To get around limitations with our hosting platform, I've set up a system whereby certain tasks can be carried out by remote machines running some special software. The way this works is that the web-based software (PHP) writes queues a job request to a database table. This request is picked up by some Windows software, which processes the job and marks it as completed. The web software has been polling the database every second or so (AJAX) to wait for the job to complete (and drawing a progress bar to keep the user happy), at which point it redirects, picks up the result of the job and carries on as normal. This works well; small amounts of data are passed in the job request itself as arguments or results text fields, and larger chunks of data are transferred using FTP. The only issue I have with the arrangement is picking up the jobs. Currently I poll the remote database server once every couple of seconds, which feels very bad. Unfortunately, the computer that is running the software is not publicly visible on the internet, hence the database workaround. I had thought of possibly having a variable rate polling - it idles at once every thirty seconds, but when it notices someone is using the modules that rely on this external software, it could increase its rate. This still isn't ideal, though. (I know the correct answer to this is to sort out some sensible hosting for both the web app and the Windows app, but sometimes you have to try and make do with what's available to you). [smile]

Quote:Original post by Kylotan
If the job producer cannot communicate with the job consumer, and the intermediate database cannot communicate outwards, then doing better than polling has to be impossible. Right?

Thank you for your affirmation that I haven't missed a trick to make this slightly less ghastly. [smile]

Quote:Possibly one optimisation is for the job consumer to post estimated completion times to the db, so that the job producer can cut back on the amount of polling it does to collect the results.

True; though the jobs usually only take between 5 and 10 seconds to complete, so I'm not sure there'd be a significant reduction in overhead.