Science a GoGo's Home Page Forums General Discussion General Science Discussion Forum Proofs of Reinforcement Learning Algorithms

Hi,

I'm suggesting a new reinforcement learning algorithm applied in my case on robots, please see: http://www.compactech.com/kartoun/articles/html/Kartoun_RA_2005_September_3_2005_Accepted.htm

I would like to describe the algorithm more scientifically; define it mathematically much better than described in the paper.

I'm asking for guidance of how to prove an algorithm, for example in the form of convergence or superiority. How can I demonstrate advantages or disadvantages of an algorithm mathematically? How can I prove convergence or divergence? How can I show if it is better or worse than other algorithms?

I've already tested the algorithm on a mobile robot for navigation.

I intend applying it for the task of finding optimal grasping, lifting and shaking policies of suspicious bags (contain anthrax, Ebola microbes or SARS), please see an initial experiment:

http://www.compactech.com/kartoun/videos/Uri_Kartoun_Plastic_Bag_Experiment_January_2_2006.wmv

Thanks a lot!

Very interesting. Any takers?

Thanks. What do you mean by "takers"?

Thanks. What do you mean by "takers"?

Hi Kartoun,

Amaranth was asking if anyone is willing to contribute to this thread.

Blacknad.

Thanks Blacknad. I appreciate your clarification.
Uri.

Kartoun,
I just watched the bag video. Wouldn't it have been smarter to have two robot arms that worked together to tear the bag apart?

Rob,
Yes, it will be much smarter - one robot holds a bag and another one tears it.
I'm considering this idea by using the Motoman Up6 robot (shown in the video) and with another robot, shown here:
http://www.compactech.com/kartoun/videos..._March_2003.wmv
Thanks,
Uri.

Err, just a thought, but surely vigorously shaking a suspicious bag without knowing what it contains (what if it's a bomb?) is a little risky. Is this happening in a public space, or has the robot taken the object away to a safe area?

Would a better option be to have the robots equipped with x-ray scanners, so authorities can at least have a bit of a look at the contents?

Rusty Rockets,

It is assumed that several kinds of bags are placed on a platform. All locks have been removed and latches and zippers opened (another universitiy is responsible for that). The task of the system is to empty the contents of an unknown bag onto the platform for subsequent scrutiny. It is assumed that the bag has already passed X-ray inspection to ensure the bag is not empty and does not contain obvious explosives (e.g., mines, gun bullets).

The case is just for non-convensional weapons such as Anthrax or Sars. The robot is placed in an isolated cage, so there is no danger to people.

Please read more here:
http://www.compactech.com/kartoun/articles/Uri_Kartoun_PhD_Proposal_Final_October_2003.pdf

Hello Uri, and welcome.
I love your Website, and am interested in your closed bag problem. I love your Robot with its music, but I am appalled at its bag shaking technique, an aspect I find rather frightening.

When a suspicious bag is found in a Public place its speedy isolation is the most important factor.
More important than its X-ray inpection, I would think?
Since you say that the Robot (and the bag?) are in an isolating cage, why waste time on X-rays?
There are other types of non-chemical explosives that would be more suitable for dispersing Biological germs and Anthrax spores. I am thinking that a standard pressurised container, mis-labelled as a room deoderant or shaving foam, could disperse virulent germs without arousing suspicion,....even if X-rayed.
My suggestion would be to dispense with X-raying and solve the immediate problem of opening the closed bag.
When the robot grasps and picks up the bag, or back-pack, why could'nt a serrated sawing knife, cut thru the bag material immediately below the Robots jaws? This would allow the bag to fall, exposing a rough semi-circular entrance into the bag.
Your Robot would again pick up the bag....should nothing fall out, the cutting would be repeated, etc. Once the bag was emptied, inspection of contents by a human.
Possible removal of zippers and clasps not required, less contamination than shaking, and a whole lot quicker.

Why don't they just call in a team that brings a container of liquid nitrogen with them, and then just drop the suspicious bag into the nitrogen. -(Could this incapacitate explosives??)

-Sorry to stray off topic but since we are on this subject...

With liquid Nitrogen you would not be able to separate the items within the bag, nor open the bag without shattering bag and contents.
Liqiud oxygen would be cheaper, but a resultant explosion, if any, would be far greater.
Dip a Q-tip size piece of cotton wool into water, then freeze in liquid oxygen. If you subject that to a hot bunsen burner or better, an oxy-acetaline torch....You will get a large explosion, enough to crack all the windows in your Laboratory.
Incapacitate Explosives you ask?
....even with a larger detonating heat input? I am not sure, sorry.

Rob ... detonation is not conflagation. All the liquid nitrogen in the world wouldn't make much of a difference.

Mike,
Thanks for your complements. The music is of "The Dresden Dolls", a band from Boston that I like.

Don't be so scared of this robotic application - all experiments are performed in a lab and I don't use any real chemical or biological materials as bag contents.

Since I'm working with a fixed arm robot (see: http://www.compactech.com/pics/motoman/ ), I'm not dealing with a suspicious bag found in a public place. An example application might be a robot located at an airport or a military base.

It is assumed that many bags a day pass X-ray test (let's say 10,000 a day). Only few of them are considered as "probably containing contagious materials". For these bags, further inspection is necessary. The inspection should be applied in a special isolated room to avoid the spread of a contagious disease (it can happen if the bag is open, emptied, and really contains contagious materials). Infection of hundreds of people can happen in minutes.

Don't mention the room deodorant and the shaving foam as an ideas to bin laden's group. They might consider it as a good idea. But seriously, you are right. There are more and more ways of damaging the public health and many of the terrorists are very sophisticated and well educated. I'm not developing the optimal system, just trying achieving a small advancement. My main contribution is the learning algorithm - shortly, the robot learns by interacting with the environment using reinforcement learning. If it reports that its learning performance is not significant it asks for human intervention for suggesting alternative solutions.

It's an idea - using a serrated sawing knife, but, I don't want to harm the bag.

Thank you so much for your response.

Uri.

Well done kartoun. A perfect example of how a sentient mind is capable of defeating the typical government-drone mentality.

When the police-brain concocts a way to stamp out prostitution, they've been trying for how many thousands of years? Or drugs, Or gambling, or whatever I might be willing to pay them some mind.

You simply, and convincingly, demonstrated by simple solutions to complex problems never work.

Reminds me so much of what happened during my last flight back to Seattle from Tokyo: I was in business class. Between take-off and landing we received not one (1), not two (2), but five (5) metal knives.

DA Morgan,
Are you cynical? If not, I'm flattered.
From Seattle to Tokyo? At what year? Who were these people? Was it for real? or just a false alarm?
I guess that, at least the food was good there at the business class... have never been there before...
Uri.

"Rob ... detonation is not conflagation. All the liquid nitrogen in the world wouldn't make much of a difference."

I was thinking it could freeze the circuitry.

Sarcastic and cynical. I wasn't so much complimenting you, though you may take it that way, as pointing out how moronic are govenment attempts to solve complex problems with simplistic solutions.

Lets be honest here ... Israel is not without its justifications but the policy over the last 40 years has hardly been inspirational when one views the results to date.

August 2005. Northwest Airlines. Do you want the flight numbers? Food was quite good with a choice of Japanese or American ... choose Japanese if you don't want the knives.

Rob ... electrons don't even freeze in liquid helium. And these days everyone uses digital clocks.
What you could potentially accomplish was stopping the generation of electricity from a battery.

But no doubt before you ever got the LN2 ordered 150 people would have published the workaround.

There is only one solution to terrorism that will work. Eliminate the social, political, and theological manufacturing plant in which it is created. Have no doubt the most recent so-called democratic election in Egypt has produced hundreds if not thousands new volunteers. And the world's governments, those that will be their victims, looked on and said nothing.

DA Morgan,

I don't think that the Israeli government is aware of my robot experiments.

Here is a nice pic of Japanese knifes:
http://hocho.o-1.jp/images/types.gif

With these knifes the cook prepares the sushi:
http://watanabeblade.com/english/pro/kurodeba3.JPG

Uri.