Sign in to follow this  

Pretend to be a browser: log in to various network sites

This topic is 2846 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

A client of me is a music band who try to do a lot through the web. They are registered at Facebook, Twitter and MySpace mainly (some other kind of sites too). Whenever they have a new event coming, they'd have to post it at all these sites. This is time consuming and error prone. What I am looking for, is a solution where I write an application (C++ or PHP) that can connect to a website and pretend to be a legitimate user. It logs in and posts a new event, news item or music file etc. There are existing (partial) solutions where RSS is involved (or something else). These are nice but not sufficient. The content that gets passed to Twitter from Facebook for example is too short for us. Also, some parts of the social sites do not allow to be fed with RSS. So I think building a HTTP connector myself (through some library) would give most possibilities. I have tried libCurl in the past with PHP. It however seems that many of these sites have measures to prevent 'bots' to connect to their sites. Also, Facebook mostly works with AJAX or JS, this will also be a problem I suppose. Has anyone done something like this? Any suggested library to use for this? How to pretend to be a browser? How to update profiles like a regular web user would? Thank you in advance!

Share this post


Link to post
Share on other sites
Perl script using LWP. The LWP::UserAgent doesn't execute javascript, but maybe you can somehow find a way to bypass the scripts (e.g. figure out URLs to post to directly instead of going to the page that a human user would go to, etc).

If that's not sufficient there's Win32::IE::Mechanize which uses MSIE to handle JS, or javascript::SpiderMonkey. Haven't used them myself so I don't know how well they work.

Share this post


Link to post
Share on other sites
There are a few ways.

First, most sites *can* be fooled by wget/curl/LWP::Robot etc. simply by specifiying approbiate headers.

Second, many of the popular social sites provide official API's to their site.
For an example, Facebook's seems to be sufficient for your needs: http://wiki.developers.facebook.com/index.php/API#Publishing_Methods
Third, you could try to use an automating Firefox plugin.

Share this post


Link to post
Share on other sites
I have done this in the past when writing a personal program to login and scrape a few secure financial sites I used. I was able to do it in C/Cpp with wininet.h functions fairly easily.

One thing that probably would be great for you would be to check out HTTP Analyzer, using the Post Data tool.

This was invaluable to me when I started trying to do this. It will watch your internet activity and when you login or post something to a page it shows you exactly what variables and values are posted to the page, so then you know exactly what to put in your post string.

I think it is something like free to try, $50 to buy, but if you're doing this for a paid job I think it would be a well justified purchase.





Share this post


Link to post
Share on other sites
Quote:
One thing that probably would be great for you would be to check out HTTP Analyzer, using the Post Data tool.


There are also Firefox addons that will do the same thing for you for free. Wireshark would another free, if less friendly, way to do it.

Share this post


Link to post
Share on other sites

This topic is 2846 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this