Does Anyone Else Data Annotate?
- Thread starter homeyg
- Start date
Comrade Ceasefire
Simmer slowly
Look to Nature for inspiration!
For example, neural networks have been around for quite a while now, and as you know they have been extremely successful for a myriad of applications. They were (and still are) justified by many researchers and practitioners as a (very crude) representation of how individual neurons work, and how collections of them operate when networked.
But real neurons and sets of neurons don't work that way at all. Back propagation is the ubiquitous algorithm in nearly all of today's machine learning. However, it is, in effect, simply the chain rule of calculus.
I highly recommend some of the youtube videos by Atrem Kirasanov on computational neuroscience.
One video came out a couple of days ago: it steps through how back propagation can be implemented.
The 2nd part will be out soon, and it will show that real neurons work very differently.
(Caveat: of course, nobody knows exactly how they do their magic, but is absolutely nothing like back-propagation!)
Training AI models using conventional techniques is not that difficult, but they can be excruciatingly slow. Worse, it is almost impossible to understand why the computational model ends up with the weights it does.
There are other methods, like evolutionary strategies, that take their inspiration from some aspects of natural Darwinian evolutionary processes like "survival of the fittest", and genetics. More elaborate models also include Baldwinian learning and aspects of population dynamics. They are, as yet, not as capable as neural networks and deep neural nets, but they have the advantage that the way they reach their solutions are less opaque.
As to computer languages, that depends on the problem at hand and how much time you are prepared to spend learning a language, writing the program you want and, most importantly, finding and correcting bugs.
Python is very popular, but it is very slow, even when combined with Cython.
For many natural language applications, LISP is a very good language, although it is probably fast falling out of fashion. (The number of brackets will eventually drive you nuts.)
My experience is in mathematical and engineering optimization: mostly ship hydrodynamics, vertical axis turbines, and wing aerodynamics. I started around 1990 when Pascal was the easiest language to use to solve simple problems on PCs. (I had to use some assembler to speed up the process. Way before that, I used a Commodore 64 and had to squeeze out an extra 2k memory by highjacking the printer's memory!)
Within about 5 years I was using Fortan and C exclusively. I still use many routines I wrote back then.
Fortran has the wonderful advantage of being backwards compatible. That is one reason it is still used today: there are many millions of lines of code that have been written and nobody is going to have the time and inclination to convert them to modern languages like
C++, Julia, Rust etc.
I use C/C++ for many programs, especially when I want to produce on-screen graphics while the program is running.
C++ is a gigantic pig of a language, but mercifully you will never need to use more than about 20% of it.
One great advantage (shared with C and Python) is that there are an enormous number of libraries you can use.
For some problems, it might be that you have to "create" your own specific "language" (probably using C and some assembler.) You'll have to be really nerdy for that. (I always have graph paper, square and triangular, in a coat pocket, but I'm not dedicated enough to start writing my own compilers.) As the eminent American philosopher Harold Callaghan once said: "A man's got to know his limitations."
If your ambition is to publish your work, then I recommend LaTeX. Nothing else comes even close. If somebody recommends Word or some other travesty, they are fools to themselves, a burden to others, and an embarrassment to their entire family. Don't be that person!
I repeat, look to Nature for inspiration for your particular problem.
How would a colony of ants solve a similar problem?
Are there lessons to be learned from the hunting behaviour of wolf packs, or the way that bee colonies swarm?
Why do blood vessels branch the way they do, and why is there a 2/9ths power law underlying their quantity and distribution in mammalian tissue?
When making honey on toast, watch how the blob drips slowly down, then continues to accelerate. Now imagine it upside down, the blob exploding up and away from the knife and how the rate is similar to that when a volcano erupts suddenly.
Try to think like the guy who was annoyed at grass seeds sticking to his socks, and then went on to invent velcro.
Finally: love what you are doing, and will it into action.
If you describe in more detail what you want to achieve I, and others here, might be able to give you more specific advice.
But remember what I said about LaTeX.
Good luck!
For example, neural networks have been around for quite a while now, and as you know they have been extremely successful for a myriad of applications. They were (and still are) justified by many researchers and practitioners as a (very crude) representation of how individual neurons work, and how collections of them operate when networked.
But real neurons and sets of neurons don't work that way at all. Back propagation is the ubiquitous algorithm in nearly all of today's machine learning. However, it is, in effect, simply the chain rule of calculus.
I highly recommend some of the youtube videos by Atrem Kirasanov on computational neuroscience.
One video came out a couple of days ago: it steps through how back propagation can be implemented.
The 2nd part will be out soon, and it will show that real neurons work very differently.
(Caveat: of course, nobody knows exactly how they do their magic, but is absolutely nothing like back-propagation!)
Training AI models using conventional techniques is not that difficult, but they can be excruciatingly slow. Worse, it is almost impossible to understand why the computational model ends up with the weights it does.
There are other methods, like evolutionary strategies, that take their inspiration from some aspects of natural Darwinian evolutionary processes like "survival of the fittest", and genetics. More elaborate models also include Baldwinian learning and aspects of population dynamics. They are, as yet, not as capable as neural networks and deep neural nets, but they have the advantage that the way they reach their solutions are less opaque.
As to computer languages, that depends on the problem at hand and how much time you are prepared to spend learning a language, writing the program you want and, most importantly, finding and correcting bugs.
Python is very popular, but it is very slow, even when combined with Cython.
For many natural language applications, LISP is a very good language, although it is probably fast falling out of fashion. (The number of brackets will eventually drive you nuts.)
My experience is in mathematical and engineering optimization: mostly ship hydrodynamics, vertical axis turbines, and wing aerodynamics. I started around 1990 when Pascal was the easiest language to use to solve simple problems on PCs. (I had to use some assembler to speed up the process. Way before that, I used a Commodore 64 and had to squeeze out an extra 2k memory by highjacking the printer's memory!)
Within about 5 years I was using Fortan and C exclusively. I still use many routines I wrote back then.
Fortran has the wonderful advantage of being backwards compatible. That is one reason it is still used today: there are many millions of lines of code that have been written and nobody is going to have the time and inclination to convert them to modern languages like
C++, Julia, Rust etc.
I use C/C++ for many programs, especially when I want to produce on-screen graphics while the program is running.
C++ is a gigantic pig of a language, but mercifully you will never need to use more than about 20% of it.
One great advantage (shared with C and Python) is that there are an enormous number of libraries you can use.
For some problems, it might be that you have to "create" your own specific "language" (probably using C and some assembler.) You'll have to be really nerdy for that. (I always have graph paper, square and triangular, in a coat pocket, but I'm not dedicated enough to start writing my own compilers.) As the eminent American philosopher Harold Callaghan once said: "A man's got to know his limitations."
If your ambition is to publish your work, then I recommend LaTeX. Nothing else comes even close. If somebody recommends Word or some other travesty, they are fools to themselves, a burden to others, and an embarrassment to their entire family. Don't be that person!
I repeat, look to Nature for inspiration for your particular problem.
How would a colony of ants solve a similar problem?
Are there lessons to be learned from the hunting behaviour of wolf packs, or the way that bee colonies swarm?
Why do blood vessels branch the way they do, and why is there a 2/9ths power law underlying their quantity and distribution in mammalian tissue?
When making honey on toast, watch how the blob drips slowly down, then continues to accelerate. Now imagine it upside down, the blob exploding up and away from the knife and how the rate is similar to that when a volcano erupts suddenly.
Try to think like the guy who was annoyed at grass seeds sticking to his socks, and then went on to invent velcro.
Finally: love what you are doing, and will it into action.
If you describe in more detail what you want to achieve I, and others here, might be able to give you more specific advice.
But remember what I said about LaTeX.
Good luck!
