Archived

This topic is now archived and is closed to further replies.

matias suarez

distributed processing doubts

Recommended Posts

matias suarez    122
I''m doing some research that is computational expensive, it does a lot of ANN and GA stuff so i decided to use 3 machines to do it (2 personal ones and my company provided laptop ) It is also possible for me to use some computers abroad (10k kilometers from here) I started writting some java classes to split the ANN work between computers with one ''server'' that assigns work units to each connected ''client'', the basic stuff seems to work well. Since my machines are linux and one is windows i decided to use java and sockets as the more portable solution. I was wondering if there''s some software already made for this kind of stuff and how a linux cluster will differ from this (i don''t really know what is a linux cluster and my googling didn''t provide clear info) Any help is appreciated! -Mat

Share this post


Link to post
Share on other sites
oliii    2196
I''m not too hot on parallel computing, but I remember using PVM (Paralle Virtual Machine) in my studies, a long time ago (well, in 1998), it worked seamlessly on Win32 and SGI platforms (Unix). I''m sure it came a long way since then, and you could probably check out the competition.

anyhow, this is what (''parallel computing libraries'') came up with google. Look for PVM, it should bring up some interesting results.

good luck.

Share this post


Link to post
Share on other sites
ironfroggy    122
You could also use something like DCOM (im sure someone implements it for linux), which would allow you to write the code natively.

The best advice I would give is to just be careful where you split up the ANN, because you definately want the least ammount of communication between systems possible. Maybe your work distribution code should account for that, even monitor module communication bandwidth and adjust as needed.

Share this post


Link to post
Share on other sites
matias suarez    122
But why I would use dcom if i''m coding java? i could move it to c or interface the dcom with java but java already has it''s own rmi.
Even so dcom or RMI the configuration, access rights and update is the most tedious and complicated part, i get some extra machines so now is a total of 6 machines.
That PVM stuff sounds quite interesting and worth to check it, thanks for the info!
Now i''ll start spreading my ANN around the planet!

-Mat

PS: yes, each net is completely isolated from the others and the GA part mutates the net parameters to find the optimal(s) ones for the problem, since each work unit travels with the the net (java code) and the data is always at the machine the communication requirements are low.

Share this post


Link to post
Share on other sites