Problems in artificial intelligence research?

views:

5339

answers:

+15 Q:

Problems in artificial intelligence research?

I'm currently a second-year undergraduate computer engineering student, and I'm a big fan of artificial intelligence. I've done formal research in nanotech, but in graduate school, I don't think that research experience will ultimately help me in what I want to do.

So, here are the questions I pose:

What are some good problems to explore as an undergrad or researcher with a small amount of experience?

Do you know of any scientists at research institutions who are working on anything that might be good for a novice to explore before going into AI research in this day? In other words, what should I gravitate towards so that I don't waste my time on the "vacuum tubes" of artificial intelligence?

It would be nice to get something done in NLP or computer vision (maybe using something widely used like OpenCV), but I don't have much of an idea of what I should work on. My interests are far too scattered in an unstructured manner.

If there are any other details you want that would help you understand my query, let me know. I'm basically looking for direction.

I should add that I am mostly interested in the "brain" of machines, not really the hardware, although, I'm currently a CE major. I'm mostly a CE major because I want to know how everything works on a machine without counting out the intricacies of hardware. I rationally assume that such a broad background will help me with future research problems. I remember reading a few papers where hardware was focused on in optimization.

+7 A:

I'd ask one of your professors, preferably somebody who is supporting research in the area. You might get some points out of it.

I'd say that a lot of wonderful AI thought from the 80s was ahead of its time, because the computers weren't powerful enough, memory not large enough, and networks not fast enough to implement the theories.

When you look at how far something like Doug Lenart's Cyc has come, you realize that it's the accumulation of 20 years of work and hardware finally catching up.

I wonder how many other AI technologies are finally coming into their own for the same reason.

Another change is the effect of statistics and having lots of data available. If you think about what Peter Norvig was able to do with statistics for his spelling corrector, and imagine how difficult that would be to implement using Rete forward induction and rules, you realize that there's a whole other avenue open to you for attacking problems.

duffymo 2010-08-29 22:24:45

Excellent advice. You propose some great points to think about. Perhaps I should dig deep into some of Norvig's textbooks. I've been meaning to do that, actually. :-)

Mr_Spock 2010-08-29 22:31:55

"I wonder how many other AI technologies are finally coming into their own for the same reason" Little if at all. AI seems to have died right after the DOD pulled the plug over lots of its AI-related projects. I've been very much interested in this area (having written a NL interpreter as my past 6th form project project ;) ). My impression while writing it was that even if modern tools exist to achieve realistic AI, there are few practical uses so far. My interests spanned especially to gaming AI...where they really didn't do any significant strides, at least, as much as I can tell.

Christian Sciberras 2010-08-29 23:18:03

@Christian - There are plugs pulled by the DoD on AI projects? Interesting. I figured they were encouraging AI advancement for weaponry.

Mr_Spock 2010-08-29 23:50:31

There was a theory, back around 1990, that AI had been not been getting increases in computer power. When first launched in roughly the early 1960s, it had great promise, and researchers got a lot of access to really powerful computers. When this didn't pan out, AI researchers got worse and worse machines, until around 1990 when it was mostly done on individual workstations, which were about the power of the early supercomputers being used. Since then, available systems have improved, and my phone can outcompute any supercomputer from the 60s.

David Thornley 2010-09-03 16:45:25

@Christian: I've been doing Google searches on my iPhone mostly by saying the search phrase into it, watching the correct phrase come up in the search box (despite noise, homonyms, and the like), and then watching relevant results appear. That's a practical use, even if it doesn't count as AI any more.

David Thornley 2010-09-03 16:53:48

@Mr_Spock - Even though they keep improving AI in, eg, weaponry, defence and the like, the original spark that led the research into AI died midway. There are several articles about this, just Google around.@David - I'm unsure what you are talking about. I've written auto-suggest scripts, sometimes that work better than Google's (given a different work environment), yet I'm no skilled AI expert. So the question is, is what Google is doing real AI? I mean, how do you put forward a practical example of something completely unrelated?

Christian Sciberras 2010-09-04 20:36:47

There are so many problems still left in AI it's really hard to suggest any specific area to work on. A better approach might be asking yourself what problem you want to solve and then simply working towards that end.

Personally for me it was information overload. Eventually this will lead to a set of algorithms that can work together to pretend that they can think like a human (predict what sort of news is important/interesting). It won't be real AI, or it might, but it's still something to work on and a lot more interesting than just doing pointless research.

Swizec Teller 2010-08-29 22:25:26

Well, I certainly don't wan to reach a dead end! haha... Thanks for the response.

Mr_Spock 2010-08-29 22:32:51

+6 A:

Grab a copy of "On Intelligence" http://www.onintelligence.org/ and see if you can crack the cortical algorithm he is talking about. Solve that one, and the rest of the AI problems fall away.

See if you can solve the problem with creating the invariant representation (from the same book) that gives us the ability to transpose songs, adapt to slightly different situations, ...

Mark0978 2010-08-29 22:30:55

Wow. I liked this answer a lot. Imagine enabling a computer that isn't made of meat (a little Minsky there!) to do that!

Mr_Spock 2010-08-29 22:36:42

Start off with this talk (by the author of 'On Intelligence'): http://www.youtube.com/watch?v=G6CVj5IQkzk

Stephen Cross 2010-08-30 00:02:05

ditto on this book. Really good read

Hamy 2010-08-30 15:21:39

@Stephen CrossThats a great talk. I like where he's going with putting the emphasis back on predictive theories (As in theories that make verifiable predictions). I'm less convinced with his particular approach though. It sounds good but just like everyone else, I'd like to see him make some juicy predictions.I love TED talks =)

CptAJ 2010-08-30 19:54:37

http://arxiv.org/list/cs.AI/recent <-- arxiv can be reasonably helpful at indicating the types of things people are talking about in a field

jkerian 2010-08-29 22:46:46

Thank you very much for this resource!

Mr_Spock 2010-08-29 22:48:55

+1 A:

You were right in your first post. You interpreted my post correctly, I mean.

OK.

On another topic, I have no training with AI (or with CS).

Anyway, someone (that my boss was associated with) invented a new way to measure the heart (to get some kind of signal from the heart): and wanted to develop a correlation between these signals (this new type of signal) and various heart conditions.

So, assuming that you were a research doctor, and had many available patients with known medical histories/diagnoses, and a new medical device which was giving you a new type of signal, then you could have the following:

For each patient
- A new signal (similar to an electrocardiogram, but lets say acoustic or something instead)
- Known medical history/diagnosis

Then your job is to discover characteristics/parameters of the new signal, which correlate with various medical conditions.

Given a new theory (a new algorithm, to correlate the new signal with existing diagnoses), it would then be easy to test:

Get new patients with existing diagnoses
Use the new instrument to get signals from these patients
Use the new algorithm to generate diagnoses from these signals
Compare these new diagnoses with the existing (know-good) diagnoses

My questions are, is this is a solvable problem, and if so then how to go about solving it?

ChrisW 2010-08-30 00:18:02

Likely solvable, at least to some extent. Back when I was in grad school, this is the sort of thing neural nets were often used for (expert systems simply codify human understanding), but there's lots of stuff for that now. Find somebody at a local U who studies machine learning, or, failing that, any sort of AI. Most academics will be happy to spend a few minutes talking and offering pointers, although for anything much longer they might charge.

David Thornley 2010-09-03 16:48:41

The problem is that AI doesn't really mean anything, particularly without an actual problem domain.

I'd start by picking a problem and then attempting to solve it with machine learning techniques.

The appropriate techniques will differ with the problem.

Brian 2010-08-30 10:29:48

The problem is that I don't know of a problem to work on or approach. That's the whole basis for my questions. :-P

Mr_Spock 2010-08-30 10:52:09

For a start, AI includes all the algorithms covered in http://aima.cs.berkeley.edu/

Jose M Vidal 2010-09-01 20:22:31

AI is fundamentally the study of things that are hard for a computer. Once computers can do them reasonably well, those things are no longer AI. That's one big reason why AI looks theoretical and impractical; another is that intelligence is really, really complicated, and it's very difficult to equal something much smarter than an insect.

David Thornley 2010-09-03 16:51:27

+14 A:

To outsiders AI seems like a specific topic. It's not. AI covers a huge swathe of techniques and applications: Think about what it means - it refers to anything involving intelligence.

I'll describe some of the major research areas and give some terminology (in no particular order). First, COMPUTER VISION. This covers everything to do with image processing, 2d and 3d geometry, shape recognition, object detection, segmention, tracking and so on. A good tutorial is at:

http://homepages.inf.ed.ac.uk/rbf/CVonline/

Second, MACHINE LEARNING. This topic covers methods of approximating functions, making decisions, classification and so on. It would include artificial neural networks and many other biologically-inspired techniques. This area was very popular in the 90's but don't write it off. A lot of biologically-inspired or grounded research is occurring in this area.

Third, SEARCH and OPTIMIZATION. Some people say all AI is search. Search is about finding solutions in highly complex (combinatorial) data, or analysis of huge volumes of data. There is a very blurry border between search and "non-AI" parts of CS. In general there is a lot of overlap with CS. Search took off with the Internet and massive storage / data acquisition, but techniques have not markedly improved in years. You may have heard about recent controversy regarding P != NP ?

Fourth, ESTIMATION - these techniques, for which you need a grounding in probability theory include Kalman Filters, Particle Filters, etc. These techniques are used to estimate variables given uncertain information. This area was responsible for the biggest leaps forward in AI between ~2000 and ~2005, especially in robotics. Include object tracking, 3-d mapping, SLAM (simultaneous localisation and mapping) in this domain.

Fifth, MODELLING, SIMULATION & DATA MINING - there is great overlap with statistics and with machine learning, but generally we're talking about modelling data, dimensionality reduction and prediction. This area also overlaps with machine vision as this sort of data often ties in with GIS (Geographic Information Systems) and other geographical / physical / environmental datasets.

Major, current applications for AI typically combine the latest techniques from all these fields and include:

Biomedical devices (esp. human assistive technologies)
Smart imaging systems on mobile devices
[increasingly] Autonomous vehicles / agents
Environmental Modelling / mapping / prediction
Financial risk & many more

There are some so-called "dead ends" - such as Expert systems - that are still ticking along, and as some have said, may well pick up again as technologies change and gaps emerge.

There is more to AI than you could possibly learn in a lifetime! Enjoy the ride. It's a good career if you get into it.

e.g. My study years: 3 yrs BSc Computer Science & Artificial Intelligence 3 yrs PhD Machine vision / Autonomous topological mapping & navigation

Work years: 1.7 yrs Financial AI (anti-money laundering) 2.3 yrs Optimization AI (logistics, vehicle routing, timetabling etc). 3.5 yrs Computer vision (tracking, security, surveillance, biomedical) 1.0 yrs GPU (CUDA) (cell-tracking, numberplate recognition, face detection/recognition etc.)

I don't regret it.

davidra 2010-08-30 11:08:13

Wow! You know a lot, and it's obvious. I value your response immensely. Thanks a lot. I don't even really know how to respond to your post appropriately. I understood everything you said (thanks to all of the interest-driven reading), but I don't know how to respond in a technical sort of way.Again, thanks a lot for this great response. This certainly helps tremendously.

Mr_Spock 2010-08-30 11:16:11

No problem! I think continuing to read about AI is an appropriate response :) PS I also recommend the Norvig textbook. PPS OpenCV is good but make sure you understand the functions properly, it's worth it.

davidra 2010-08-30 11:27:11

I think I'll purchase the textbook very soon! Thanks for the suggestions!

Mr_Spock 2010-08-30 12:51:02

+1 - This is a great answer: well informed. I second the Peter Norvig text recommendation.

duffymo 2010-08-30 16:32:49

I find your definition and explanation of machine learning quite wrong. Machine learning is *not* biologically inspired algorithms, though the set of biologically inspired algorithms is considered to be the old part of it. Neural networks and genetic algorithms and such are old though still under research. The dominant fields in ML are kernel based algorithms (e.g. SVM) and Bayesian algorithms like Gaussian Processes and Probabilistic networks, which are not biologically inspired. I wonder where you group things like Computational Learning Theory, and in what category is the netflix prize.

Muhammad Alkarouri 2010-08-30 18:03:49

Neural networks are inspired by biological systems. The threshold system clearly shows that. Compare TLUs to neurons. Machine learning itself isn't completely biologically inspired (I guess), but there are components that certainly are.

Mr_Spock 2010-08-30 19:51:11

Yes. Neural networks are biologically inspired systems, and almost 15 years out of date. Can you show me any current research in neural networks? Even journals like IEEE Neural Networks and conferences like NIPS don't do neural networks anymore. **Nobody** currently does TLUs or neurons. Those were in in my undergrad project in 1999, and I was out of date at the time.

Muhammad Alkarouri 2010-08-30 19:58:16

Most of the IEEE Neural Network publications are estimation techniques and data mining. Most of NIPS (the Neural Information Processing Systems conference) is kernels, graphical models, information theory and Markov models. Have a look at [Journal of Machine Learning Research](http://jmlr.csail.mit.edu/papers/v11/). In the last volume (2010) I cannot find a single paper about neural networks or other biologically inspired systems.

Muhammad Alkarouri 2010-08-30 20:09:47

Yes, I can show you current research in neural networks. Look at the Neural Network Research Group (NNRG) at UT Austin or the Evolutionary Complexity Research Lab (EPlex) at UCF. In fact, the entire HyperNEAT project has been extremely successful, resulting in papers at AAMAS and AAAI in the last couple of years. A recent paper from JMLR (2010): http://eplex.cs.ucf.edu/publications/2010/verbancsics.jmlr10.html

Wesley Tansey 2010-08-30 22:43:59

I've stumbled upon neural nets a few times in the JMLR. I agree with Wesley here. I'm no researcher yet, but I've come across ANNs in modern papers.

Mr_Spock 2010-08-30 22:55:23

@Wesley: Thanks a very interesting paper. My fault that I skimmed the title and abstract and filed it mentally in high dimensional encoding rather than evolutionary algorithms. My point wasn't that neural networks are not used; I mentioned in my first comment _still under research_. Though I have to say that most research there has very little to do with the biological origins. The current papers wouldn't go about TLUs and neurons. Do you accept the description of machine learning in the answer above? Excluding kernel and Bayesian methods and dimensionality reduction and prediction?

Muhammad Alkarouri 2010-08-30 23:51:05

@Mr_Spock: you are quite right. The operative words are "come across". ANNs aren't the majority of ML so you can't describe machine learning as largely biologicaly inspired systems. Machine learning is mostly pattern recognition, or can be defined as algorithms that improve with experience. I am currently a researcher in Chemoinformatics, and the machine learning algorithms we use are largely graph theory inspired.

Muhammad Alkarouri 2010-08-30 23:56:42

@Muhammad: Interesting. I should probably teach myself as much as I can about the mathematics of combinatorics/graph theory/statistics before going into the actual AI research, eh?

Mr_Spock 2010-08-31 01:36:20

@Mr_Spock: The more math you have the better your chances, yes. But don't let it stop you from doing AI research as soon as you want. Chances are that your current level of math is enough to do a lot of AI already.

Muhammad Alkarouri 2010-08-31 02:42:30

@Muhammad: I think it's just that the ANN field has advanced so far that ANN research has become a fuzzy category. For example, does Geoffrey Hinton's work on deep belief nets count? The AI community is really split into two groups. The mainstream group works, like you mentioned, on Gaussian processes, Statistical Relational Learning, etc. The black sheep group typically works on bio-inspired algorithms. The origins of the division go back too far to get into here. The groups are slowly merging again as the old guard fades, but for now it's still there. I think both groups are valid though. :)

Wesley Tansey 2010-08-31 18:49:45

@Wesley: deep belief nets are a remarkable framework and they certainly count. The interaction of ANN with probabilistic and high dimensional modelling has certainly given a lot in interesting work. I was just too sensitive to the fact that out of the frameworks mentioned [here](http://hunch.net/?p=224), every one of them is not machine learning but is in some other classification (searching, modelling, optimisation) by the definitions in the answer above. In short, Hinton's work would be the _only_ valid machine learning approach in the ones you mentioned.

Muhammad Alkarouri 2010-08-31 20:22:26

I would suggest the book Programming Collective Intelligence. This is useful to introduce you to a set of problems revolving around document mining and collaboration filtering. It has Python code and would introduce you to the problem and possible approaches rather quickly. Once you are familiar with the problem, however, you will outgrow the book fairly quickly. Use any one of the excellent references in the other answers.

If you are looking for a bigger challenge then check Geometry and Meaning by Dominic Widdows. It is a brilliant tour of relations between mathematical concepts and language ones, based on Widdows experience of the Infomap NLP project in Stanford. If you become more interested in that then of course there are the research papers published by the group.

I generally prefer problem oriented research, so your approach is the right one.

Muhammad Alkarouri 2010-08-31 00:44:20

Thank you VERY much for this, Muhammad. This really helps. I'll be checking this book out as soon as possible. It's funny -- MIT has a research facility dedicated to collective intelligence, and I was just looking into it a few months ago. Thanks for the resource!

Mr_Spock 2010-08-31 01:57:10

You are welcome. Glad to have a positive contribution. Good luck!

Muhammad Alkarouri 2010-08-31 02:46:03

+1 A:

There was some AI research that appeared in Science last year:

http://www.sciencemag.org/cgi/content/abstract/324/5923/81

It might give you an idea of some very recent advancements.

Inverse 2010-09-03 16:32:16

ansaurus

tags:

views:

answers:

Problems in artificial intelligence research?

related questions