Hidden Limits of Machine Learning

Note: Machine learning(ML) is a subset of Artificial Intelligence(AI), AI includes, for example, evolutional programming, swamp intelligence etc. In other words Machine learning is a collection of algorithms and techniques used to design systems that learn from data.

The lure of Machine Learning methods has been overwhelming. The range of Machine Learning applications is wide from Detection of Credit Card Fraud to Classification of Biological Particles in Electron-microscopy Images. People are getting to believe that Machine Learning is Universal Tool capable to solve any classification problem (or at least with 99% reliability)

The purpose of the given note is to add a new argument against of Machine Learning’s omnipotence. It’s worth mentioning that there are theoretical limitations on Machine Learning, e.g. Cybenko’s Theorem for Neural Networks.

To warm up let’s consider a classification (or separation) of two-dimensional classes (objects) with the following restriction on the objects: the ‘amount’ of boundary points is negligible compared with the ‘amount’ of interior points (generally, one can easily create a weird object where boundary points are 99% of all points, we omit these cases).
So, for two-dimensional classes even if some of the classes touch each another and even if class boundaries are totally fractal then a non-linear separation curve (or set of curves) can be found with low percentage of false classifications. False classifications are expected to happen if some classes touch each another (however due to the restriction the percentage of false classifications tends to be low since boundary samples consists a small percent).
Let’s for the sake of simplicity consider Two Classes Classification Problem. Moreover, let’s assume that each class (object) represents a region in N-dimensional space (where N is much more one). Let’s also assume that each object is close to a N-dimensional sphere of radius R and the two objects touch each another at some boundary segment.
Volume of N-dimensional sphere of the radius R:

           Vol(R) =  πN/2 RN  / Γ(N/2+1)

         where Γ is the gamma function.
Then the volume of near-surface “peel” of thickness ε (ε<R):

Vol(R) – Vol(R- ε) =   πN/2 RN  (1 –  (R – ε)N/RN) / Γ(N/2+1)

For ε=0.1 R the ε-peel’s volume equal to πN/2 RN (1 –  0.9N) / Γ(N/2+1). Hence, when N is much greater 1 then the volume of 0.1-peel is close to Vol(R). In other words, almost all mass of N-dimensional sphere (for N much greater than 1) is concentrated at the thin 0.1R peel. Actually the interior of this sphere is almost ‘empty’.

Consequently, if the two classes touch each another along some boundary, for example touching boundary is 10% of the whole boundary) then the separation curve would mismatch roughly 50% of boundary points. Hence, the total number of false classifications is about 10%.

This is an interesting property of high-dimensional objects is that most of their volume is near the surface.

 

Expert Systems

Fundamental problem in development of expert systems is not in techniques of inference but in experts themselves. Because human experts are not able to articulate their thinking process to codify it.

 

Note:

In Machine Learning the concept “machine” is not necessarily a collection of physical parts (e.g. bearings, shafts), but any entity which does a specific job.

193 Responses

  1. You could certainly see your expertise within the work you write. The arena hopes for even more passionate writers such as you who are not afraid to say how they believe. All the time go after your heart.

  2. I’d have to examine with you here. Which is not one thing I usually do! I take pleasure in reading a post that may make folks think. Additionally, thanks for permitting me to comment!

  3. There are certainly a variety of details like that to take into consideration. That is a great level to carry up. I offer the ideas above as general inspiration but clearly there are questions like the one you deliver up the place the most important factor will probably be working in sincere good faith. I don?t know if finest practices have emerged around things like that, but I’m positive that your job is clearly identified as a fair game. Each girls and boys feel the impression of just a moment’s pleasure, for the remainder of their lives.

  4. What i don’t realize is in fact how you are now not actually much more well-preferred than you may be right now. You’re so intelligent. You recognize therefore considerably in the case of this matter, made me in my opinion imagine it from so many varied angles. Its like women and men aren’t interested until it is one thing to accomplish with Woman gaga! Your personal stuffs great. Always take care of it up!

  5. Awesome website you have here but I was wanting to know if you knew of any message boards that cover the same topics discussed in this article? I’d really like to be a part of group where I can get opinions from other experienced people that share the same interest. If you have any recommendations, please let me know. Cheers!

  6. Thank you for posting this. I really enjoyed reading it, especially because it addressed my question. It helped me a lot and I hope it will help others too.

  7. Thank you for posting this. I really enjoyed reading it, especially because it addressed my question. It helped me a lot and I hope it will help others too.

  8. With every little thing that appears to be developing inside this specific subject material, a significant percentage of opinions happen to be fairly radical. Nevertheless, I am sorry, but I can not give credence to your entire theory, all be it stimulating none the less. It would seem to me that your remarks are not entirely validated and in actuality you are yourself not even fully certain of the assertion. In any event I did appreciate examining it.

    1. i suggest considering this post as an heuristic. A baseball player don’t solve a system of differential equations to determine where and when the ball fell. He uses a heuristic rules, e.g. keeping same angle to the moving ball.

  9. Magnificent beat ! I would like to apprentice at the same time as you amend your website, how can i subscribe for a weblog web site? The account helped me a applicable deal. I had been a little bit acquainted of this your broadcast offered brilliant transparent idea

  10. Hi there I am so excited I found your blog, I really found you by accident, while I was researching on Aol for something else, Nonetheless I am here now and would just like to say thanks for a marvelous post and a all round exciting blog (I also love the theme/design), I don’t have time to go through it all at the moment but I have book-marked it and also added in your RSS feeds, so when I have time I will be back to read a lot more, Please do keep up the great job.

  11. I like what you guys are up also. Such smart work and reporting! Keep up the excellent works guys I have incorporated you guys to my blogroll. I think it’ll improve the value of my site 🙂

  12. Unquestionably believe that that you said. Your favorite reason appeared to be on the web the easiest thing to be mindful of. I say to you, I definitely get annoyed whilst people consider issues that they just do not realize about. You controlled to hit the nail upon the top and outlined out the whole thing with no need side-effects , folks can take a signal. Will likely be back to get more. Thank you

  13. You really make it seem so easy together with your presentation but I to find this topic to be actually something that I feel I would never understand. It kind of feels too complex and very extensive for me. I’m having a look forward to your subsequent publish, I?ll attempt to get the dangle of it!

  14. I’ve observed that in the world the present moment, video games would be the latest trend with children of all ages. Many times it may be out of the question to drag your children away from the video games. If you want the very best of both worlds, there are various educational activities for kids. Interesting post.

  15. The very heart of your writing whilst appearing agreeable at first, did not settle perfectly with me after some time. Someplace within the sentences you managed to make me a believer unfortunately only for a very short while. I nevertheless have got a problem with your leaps in logic and one would do well to help fill in those gaps. When you can accomplish that, I could undoubtedly end up being impressed.

  16. Good post. I study something more difficult on different blogs everyday. It’s going to at all times be stimulating to learn content material from different writers and observe a little something from their store. I?d want to use some with the content on my weblog whether or not you don?t mind. Natually I?ll offer you a link in your internet blog. Thanks for sharing.

  17. It’s the best time to make some plans for the future and it is time to be happy. I have read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you can write next articles referring to this article. I desire to read even more things about it!

  18. With havin so much written content do you ever run into any problems of plagorism or copyright violation? My blog has a lot of unique content I’ve either written myself or outsourced but it seems a lot of it is popping it up all over the internet without my permission. Do you know any techniques to help reduce content from being ripped off? I’d genuinely appreciate it.

  19. Throughout this grand design of things you receive an A just for hard work. Where exactly you actually misplaced everybody was first on all the details. You know, people say, details make or break the argument.. And that could not be much more true here. Having said that, permit me say to you just what exactly did do the job. Your article (parts of it) can be extremely persuasive and that is probably why I am taking the effort to opine. I do not really make it a regular habit of doing that. 2nd, while I can see a leaps in reasoning you make, I am not really sure of how you appear to connect the points that make the actual conclusion. For the moment I will subscribe to your point but wish in the future you actually link your dots better.

  20. I?ve been exploring for a little bit for any high quality articles or blog posts on this kind of area . Exploring in Yahoo I at last stumbled upon this web site. Reading this information So i?m happy to convey that I have an incredibly good uncanny feeling I discovered just what I needed. I most certainly will make certain to do not forget this website and give it a look regularly.

  21. Somebody essentially help to make significantly articles I’d state. That is the first time I frequented your website page and to this point? I amazed with the research you made to create this particular submit extraordinary. Fantastic job!

  22. Thank you for your post. I really enjoyed reading it, especially because it addressed my issue. It helped me a lot and I hope it will help others too.

  23. Oh my goodness! an incredible article dude. Thank you Nevertheless I’m experiencing problem with ur rss . Don?t know why Unable to subscribe to it. Is there anyone getting similar rss problem? Anyone who knows kindly respond. Thnkx

  24. Thank you for your post. I liked reading it because it addressed my issue. It helped me a lot and I hope it will help others too.

  25. Thank you for posting this post. I found it extremely helpful because it explained what I was trying to say. I hope it can help others as well.

  26. What?s Happening i am new to this, I stumbled upon this I have found It positively helpful and it has helped me out loads. I hope to contribute & help other users like its aided me. Great job.

  27. I?m impressed, I must say. Really rarely do I encounter a blog that?s each educative and entertaining, and let me inform you, you have got hit the nail on the head. Your concept is excellent; the difficulty is one thing that not enough people are speaking intelligently about. I’m very comfortable that I stumbled throughout this in my seek for one thing relating to this.

  28. Another issue is that video games usually are serious anyway with the most important focus on learning rather than amusement. Although, we have an entertainment part to keep your young ones engaged, just about every game is frequently designed to develop a specific expertise or program, such as numbers or technology. Thanks for your write-up.

  29. This is really interesting, You’re a very skilled blogger. I have joined your feed and look forward to seeking more of your wonderful post. Also, I’ve shared your site in my social networks!

  30. Nice post. I used to be checking constantly this blog and I am inspired! Very useful info specifically the final phase 🙂 I care for such information much. I was seeking this certain information for a very long time. Thank you and best of luck.

  31. Thanks for your write-up on this website. From my own personal experience, often times softening right up a photograph might provide the photo shooter with a dose of an inspired flare. Often however, that soft cloud isn’t exactly what you had under consideration and can usually spoil an otherwise good image, especially if you intend on enlarging this.

  32. This website online is known as a stroll-through for the entire info you needed about this and didn?t know who to ask. Glimpse right here, and also you?ll undoubtedly uncover it.

  33. That is the correct weblog for anybody who wants to search out out about this topic. You realize a lot its almost hard to argue with you (not that I really would want?HaHa). You positively put a new spin on a subject thats been written about for years. Great stuff, simply nice!

  34. It?s really a nice and useful piece of info. I am glad that you shared this useful info with us. Please keep us informed like this. Thank you for sharing.

  35. I have really learned some new things by your blog site. One other thing I want to say is the fact that newer computer os’s have a tendency to allow a lot more memory to be played with, but they additionally demand more storage simply to function. If an individual’s computer is not able to handle additional memory plus the newest software package requires that memory space increase, it usually is the time to shop for a new Computer. Thanks

  36. Thank you for this article. I’d also like to say that it can become hard while you are in school and merely starting out to create a long credit score. There are many pupils who are only trying to make it and have an extended or beneficial credit history can sometimes be a difficult issue to have.

  37. Somebody necessarily lend a hand to make critically articles I’d state. This is the first time I frequented your website page and so far? I amazed with the research you made to create this particular submit amazing. Great process!

  38. Great work! This is the kind of information that are meant to be shared across the net. Disgrace on Google for now not positioning this publish upper! Come on over and discuss with my site . Thank you =)

  39. F*ckin? awesome issues here. I?m very glad to peer your article. Thank you so much and i’m having a look ahead to touch you. Will you kindly drop me a e-mail?

  40. A person necessarily assist to make significantly articles I would state. This is the very first time I frequented your website page and so far? I surprised with the research you made to make this particular submit amazing. Fantastic job!

  41. Definitely believe that which you said. Your favorite justification appeared to be on the web the simplest thing to be aware of. I say to you, I definitely get annoyed while people think about worries that they plainly do not know about. You managed to hit the nail upon the top and defined out the whole thing without having side-effects , people could take a signal. Will probably be back to get more. Thanks

  42. you are really a good webmaster. The web site loading speed is amazing. It seems that you are doing any unique trick. Also, The contents are masterpiece. you’ve done a magnificent job on this topic!

  43. The subsequent time I read a blog, I hope that it doesnt disappoint me as much as this one. I imply, I know it was my option to learn, however I truly thought youd have one thing attention-grabbing to say. All I hear is a bunch of whining about one thing that you might repair in case you werent too busy searching for attention.

  44. Thanks for giving your ideas. Another thing is that students have an option between federal government student loan and also a private student loan where it’s easier to choose student loan debt consolidation than through the federal student loan.

  45. I’ve observed that in the world these days, video games are classified as the latest rage with children of all ages. There are times when it may be out of the question to drag your children away from the activities. If you want the very best of both worlds, there are various educational games for kids. Interesting post.

  46. I have fun with, cause I found exactly what I used to be having a look for. You’ve ended my four day long hunt! God Bless you man. Have a great day. Bye

  47. Hey very nice website!! Man .. Beautiful .. Amazing .. I’ll bookmark your web site and take the feeds also?I am happy to find numerous useful info here in the post, we need develop more techniques in this regard, thanks for sharing. . . . . .

  48. What i don’t realize is in truth how you’re no longer really much more well-preferred than you might be right now. You are very intelligent. You recognize thus considerably in terms of this subject, made me for my part believe it from numerous varied angles. Its like women and men are not fascinated except it?s something to do with Lady gaga! Your individual stuffs nice. All the time maintain it up!

  49. I have been browsing online greater than 3 hours today, yet I never found any interesting article like yours. It?s lovely value enough for me. In my view, if all webmasters and bloggers made excellent content as you did, the web will probably be much more helpful than ever before.

  50. You actually make it seem so easy with your presentation but I find this topic to be really something that I think I would never understand. It seems too complex and extremely broad for me. I’m looking forward for your next post, I will try to get the hang of it!

  51. The very crux of your writing whilst sounding agreeable in the beginning, did not really work very well with me after some time. Someplace within the paragraphs you managed to make me a believer but only for a very short while. I still have got a problem with your leaps in logic and one would do nicely to help fill in all those gaps. In the event you actually can accomplish that, I could undoubtedly end up being impressed.

  52. Wow, wonderful blog layout! How long have you ever been blogging for? you made blogging glance easy. The total glance of your web site is wonderful, as smartly as the content!

  53. Thanks a bunch for sharing this with all of us you actually know what you are talking about! Bookmarked. Kindly also visit my site =). We could have a link exchange arrangement between us!

  54. I do not even know how I stopped up right here, but I assumed this put up was great. I don’t recognise who you are however certainly you are going to a famous blogger in case you are not already 😉 Cheers!

  55. Thank you, I have recently been looking for information approximately this topic for a while and yours is the best I’ve discovered till now. However, what concerning the bottom line? Are you sure in regards to the source?

  56. Hello, i feel that i noticed you visited my site thus i got here to ?go back the favor?.I’m attempting to to find things to enhance my website!I assume its adequate to use some of your ideas!!

  57. As I web-site possessor I believe the content material here is rattling fantastic , appreciate it for your hard work. You should keep it up forever! Good Luck.

  58. I do agree with all of the ideas you’ve presented in your post. They are really convincing and will certainly work. Still, the posts are very short for novices. Could you please extend them a bit from next time? Thanks for the post.

  59. I’ve been browsing online more than 3 hours today, yet I never found any interesting article like yours. It?s pretty worth enough for me. In my opinion, if all webmasters and bloggers made good content as you did, the net will be much more useful than ever before.

  60. A formidable share, I just given this onto a colleague who was doing slightly analysis on this. And he in actual fact bought me breakfast because I found it for him.. smile. So let me reword that: Thnx for the deal with! But yeah Thnkx for spending the time to discuss this, I really feel strongly about it and love studying extra on this topic. If potential, as you turn into experience, would you mind updating your blog with extra particulars? It’s highly helpful for me. Large thumb up for this weblog submit!

  61. Its like you read my mind! You appear to know so much about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a little bit, but instead of that, this is great blog. A fantastic read. I will definitely be back.

  62. I have been browsing online more than three hours these days, yet I by no means found any fascinating article like yours. It?s pretty price sufficient for me. In my opinion, if all webmasters and bloggers made good content material as you probably did, the web can be much more helpful than ever before.

Leave a Reply

Your email address will not be published. Required fields are marked *