Sign in to follow this  
  • entries
    122
  • comments
    246
  • views
    89777

Journal Comment Eater

Sign in to follow this  

588 views

I found some free time hiding beneath my bed so I spent some 3 and a half hours making a little application which sits in your taskbar and tells you if you got new comments in your game dev journal. It was fun, Linq is my favourite part of the otherwise mediocre C#. It does this by html scraping the respective page. Every 20 minutes it checks if there have been new comments since the last check. If there has it tries to find out which posts got the comments and pops out a ballon telling you which. If there have been new entries since the last check it only checks the old entries. All this was done quite easily and not many lines of code thanks to the html agility pack and Xlinq.

Every 20 mins it checks if there was a new post at a url which is pasted in a textbox. If the textbox is empty it checks for a file located in the same directory as it for the url, if not it uses Gaiidens journal as the default.

    if (textBoxUrl.Text == "")
{
if (File.Exists(infFilePath))
{
var urlFromFile = System.IO.File.ReadAllText(infFilePath); textBoxUrl.Text = urlFromFile;
return (urlFromFile == "") ? "https://www.gamedev.net/community/forums/mod/journal/journal.asp?jn=251283" : urlFromFile;
}
else
{
textBoxUrl.Text = "https://www.gamedev.net/community/forums/mod/journal/journal.asp?jn=251283";
return textBoxUrl.Text ;}
}
else
return textBoxUrl.Text;



It then checks if there have been any new comments since the last check.

var currentCheck = (currentCheckInit.Count > checkCompare.Count) && checkCompare.Count>0  ? 
currentCheckInit.Skip(currentCheckInit.Count - checkCompare.Count).ToList()
: currentCheckInit;
var sb = new StringBuilder();
if (currentCheck.Sum(p => p.CommentCount) != checkCompare.Sum(p => p.CommentCount))
{
for (int i = 0; i < checkCompare.Count; i++)
{
if (currentCheck.IsComment && currentCheck.CommentCount != checkCompare.CommentCount)
{
var msg = "At " + DateTime.Now.ToShortTimeString() + " Found "
+ (currentCheck.CommentCount - checkCompare.CommentCount)
+ " New Comment(s) in thread, " + checkCompare[i - 1].Title;
listBoxLog.Items.Add(msg);
sb.AppendLine(msg);
newComment = true;
notifyIcon1.Text = "New Comment(s) found";
}
}
}







You can also save the last query results to disk. Then later on you may download from the site and check against disk so the program doesnt have to run continuosly to be useful.

     private void buttonSaveState_Click(object sender, EventArgs e)
{
var sb = new StringBuilder();
oldCheck.ForEach(entry => sb.AppendLine (entry.CommentCount + "|"+entry.IsComment + "|" +entry.Title));
File.WriteAllText(statePath, sb.ToString());
}

private void buttonCompareStates_Click(object sender, EventArgs e)
{
var urlFromFile = System.IO.File.ReadAllText(infFilePath);
if ((urlFromFile != textBoxUrl.Text) && textBoxUrl.Text != "")
MessageBox.Show("Warning Url found in file and in textbox do not match. This *may* cause discrapncies.",
"Are you Sure you know what you are doing?",
MessageBoxButtons.OK, MessageBoxIcon.Warning);
var tmpCheck = new List();
var dat = File.ReadAllLines(statePath).ToList ();
dat.ForEach(item => { var s = item.Split('|');
var newE = new JournalEntry();
newE.CommentCount = int.Parse(s[0]);
newE.IsComment = bool.Parse(s[1]);
newE.Title = s[2];
tmpCheck.Add(newE); });
var currentCheck = util.PollGDNEt(CheckUrlOptions());
inited = true;
DoComparison(currentCheck, tmpCheck);
}







The part that scrapes the page is here

public static List PollGDNEt(string url)
{
HtmlWeb page = new HtmlWeb();
HtmlDocument doc = page.Load(url);
var xdoc = doc.ToXDocument();

var queryResults = from element in xdoc.Descendants()
where element.HasAttributes
&& element.Name.LocalName == "span"
&& (element.FirstAttribute.Value == "regularfont" || (element.FirstAttribute.Value == "smallfont" && element.Value.Contains("Comments")))
select new {Title = element.Value,
IsComment = element.FirstAttribute.Value == "smallfont",
Count = element.FirstAttribute.Value == "smallfont" ?
int.Parse( Regex.Match( element.Value, @"\d+").Value ) : 0 };
var Entries = new List();
foreach (var result in queryResults)
{
var entry = new JournalEntry ();
entry.CommentCount = result.Count; entry.IsComment = result.IsComment ; entry.Title = result.Title ;
Entries.Add(entry);
}
return Entries;
}







Full Source is here (svn) or source rar'd. And app is here (requires .NET 3.5). Some stats in order of # comments found on page:

User | Total comments on Page | Avg Comments Per Post | Total Posts

-------------------------------------------------------------------------------------------------------
TrapperZoid 20 4 5
Drew 19 1.27 15
Talestyn 16 1.07 15
Ben 16 2 8
Ravuya 16 1.14 14
Me 6 1.2 5


Also Mike P if I you read this and I could get the source for your line counter project, would be cool.
Sign in to follow this  


13 Comments


Recommended Comments

Quote:
Also Mike P if I you read this and I could get the source for your line counter project, would be cool.


Sure thing, although you could have just PMed me :D

Actually, the source is online and downloadable via SVN, although if you're extremely lazy, you could ask me nicely and I might zip up a copy for you.

The code isn't all that pretty, and I make no promises. Right now it actually tabulates totals for all files as well as per-filter, but I have no good way of showing that right now (HtmlReport isn't finished yet), so I just print the grand totals to the console. If you fix it up, be sure to let me know.

Share this comment


Link to comment
That, sir, is an excellent idea. I've downloaded it, and only messed with it a little bit, but it seems *quite nice*. Might have to mess with it and see what else could be done with it.

Also, cookie++

Share this comment


Link to comment
Quote:
Original post by nerd_boy
That, sir, is an excellent idea. I've downloaded it, and only messed with it a little bit, but it seems *quite nice*. Might have to mess with it and see what else could be done with it.

Also, cookie++


Thanks :)

Share this comment


Link to comment
Quote:
Original post by Mike.Popoloski
Quote:
Also Mike P if I you read this and I could get the source for your line counter project, would be cool.


Sure thing, although you could have just PMed me :D

Actually, the source is online and downloadable via SVN, although if you're extremely lazy, you could ask me nicely and I might zip up a copy for you.

The code isn't all that pretty, and I make no promises. Right now it actually tabulates totals for all files as well as per-filter, but I have no good way of showing that right now (HtmlReport isn't finished yet), so I just print the grand totals to the console. If you fix it up, be sure to let me know.


I downloaded via svn so wasnt much effort so we both got to be lazy :). Firstly thanks alot Ive been looking for a line counter without all the unneccessary bells and whistles or exorbitant pricing.

So Ive gotten the program to a shape I am happy with. The config.txt was not tied down to anyone location so was saving in some random dir with no hope of my finding it. I tied it down to same dir as app. I also added a summary field to FileNode which displays all the info per file in the console and spits out to a text and csv file so nice graphs can be made. I also added optional removal of counting for things which cause bloat in the code via regular expressions contained in a filter_config.txt (for example I dont want *.designer.cs to be counted so i have a ^.*\.Designer.*$|^AssemblyInfo\..*$ filter). Finally I added the ability to ignore project files and simply recursively count all files in a folder, with counting of code per file type with percentages as well - (built on top of and not breaking your layout :) - which is useful for asp.net web sites. Necessary changes to command line were done (linecounter folderpath [/fr |/f] for example). The rar'd code changes can be found here for your review.

Sample Output
Total Lines : 749

Code Lines : 621
Comments : 2
Blank Lines : 126

------------------------

Tables.fs : Blank Lines: 13, Code Lines: 39, Comments: 0, Total Lines: 52
------------------------


------------------------

VCProject.cs : Blank Lines: 11, Code Lines: 43, Comments: 0, Total Lines: 54
------------------------


------------------------

Program.cs : Blank Lines: 13, Code Lines: 95, Comments: 0, Total Lines: 108
------------------------


------------------------

Nodes.cs : Blank Lines: 3, Code Lines: 23, Comments: 0, Total Lines: 26
------------------------


------------------------

HtmlReport.cs : Blank Lines: 17, Code Lines: 64, Comments: 0, Total Lines: 81
------------------------


------------------------

Helpers.cs : Blank Lines: 6, Code Lines: 26, Comments: 0, Total Lines: 32
------------------------


------------------------

CSProject.cs : Blank Lines: 11, Code Lines: 66, Comments: 0, Total Lines: 77
------------------------


------------------------

Counter.cs : Blank Lines: 52, Code Lines: 265, Comments: 2, Total Lines: 319
------------------------

.cs files, Count 7, Total Lines 697, Code Lines 582 [Percentage: 94 %]

.fs files, Count 1, Total Lines 52, Code Lines 39 [Percentage: 6 %]



Share this comment


Link to comment
>_> How the flippin' hoo-haa did you get your post between mine and Mike.Popoloski's!? [disturbed]

Share this comment


Link to comment
Quote:
Original post by nerd_boy
>_> How the flippin' hoo-haa did you get your post between mine and Mike.Popoloski's!? [disturbed]
Like this [smile]

Share this comment


Link to comment
Quote:
Original post by Daerax*snip*


Cool, that's the sort of thing I wanted to add, but simply didn't have time for. I still have a few other ideas, but for now it's one of the lowest priorities in my ever-growing list of projects I want to work on.

Anyways, glad it could be of some use for someone.

Share this comment


Link to comment
Quote:
Original post by benryves
Great idea for a utility! [grin] Cheers!


Thanks hah. I got a bubble stating you posted earlier heh.

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now