Jump to content
  • Advertisement
Sign in to follow this  

getting data from a webpage?

This topic is 3278 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I am trying to make a program so that whenever I visit a certain webpage, it will grab information off it and store it in a log file on my computer along with the time accessed. The problem is I have no idea where to start. I've made some simple games in C++ with SDL, but I imagine that that isn't really applicable here. Do I need to learn another language? What is the best way to approach this? Is this even a realistic goal, or am I way out of my league? By the way, the browser I use is Google Chrome, but I guess I can change if it is really important.

Share this post


Link to post
Share on other sites
Advertisement
My guess would be the easiest way is to write a plugin or extension. I don't know how well supported they are in Chrome, or what functionality you can achieve, but you could start by taking a look at these tutorials.

Share this post


Link to post
Share on other sites
you could use HTTP protocool with winsock to grab the html in to a buffer , pritty simple to form the headers and send them and get replys

Share this post


Link to post
Share on other sites
Anddos makes a valid point if your requirement of 'whenever I visit a certain webpage' is not really a strict requirement. If it isn't, writing a 'screen-scraper' can be pretty simple -- I believe Python and Ruby have libraries available. Just need to create a buffer, create a connection, grab the data, and parse what you want (typically with regex).

But if you want this action to occur based on what you do with your browser, creating an extension would probably be a lot easier than trying to hook into the browser with an external application via some other method.

Share this post


Link to post
Share on other sites
Perhaps more interestingly is that you want the logging done whenever you visit the webpage in your webbrowser. Does it have to be tied to your browser visit? Or is it just that you want the data collected periodically. That is, you would be perfectly happy to have that data without having to see the page in your browser.

(Edit: Late in posting eh, everyone covered the point.)

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!