Journal Comment Eater

posted in Journal
Published December 20, 2008
Advertisement
I found some free time hiding beneath my bed so I spent some 3 and a half hours making a little application which sits in your taskbar and tells you if you got new comments in your game dev journal. It was fun, Linq is my favourite part of the otherwise mediocre C#. It does this by html scraping the respective page. Every 20 minutes it checks if there have been new comments since the last check. If there has it tries to find out which posts got the comments and pops out a ballon telling you which. If there have been new entries since the last check it only checks the old entries. All this was done quite easily and not many lines of code thanks to the html agility pack and Xlinq.

Every 20 mins it checks if there was a new post at a url which is pasted in a textbox. If the textbox is empty it checks for a file located in the same directory as it for the url, if not it uses Gaiidens journal as the default.

    if (textBoxUrl.Text == "")            {                if (File.Exists(infFilePath))                {                    var urlFromFile = System.IO.File.ReadAllText(infFilePath); textBoxUrl.Text = urlFromFile;                    return (urlFromFile == "") ? "https://www.gamedev.net/community/forums/mod/journal/journal.asp?jn=251283" : urlFromFile;                }                else                {                    textBoxUrl.Text = "https://www.gamedev.net/community/forums/mod/journal/journal.asp?jn=251283";                    return textBoxUrl.Text ;}            }            else                return textBoxUrl.Text;


It then checks if there have been any new comments since the last check.

var currentCheck = (currentCheckInit.Count > checkCompare.Count) && checkCompare.Count>0  ?                                     currentCheckInit.Skip(currentCheckInit.Count - checkCompare.Count).ToList()                                     : currentCheckInit;            var sb = new StringBuilder();            if (currentCheck.Sum(p => p.CommentCount) != checkCompare.Sum(p => p.CommentCount))            {                for (int i = 0; i < checkCompare.Count; i++)                {                    if (currentCheck.IsComment && currentCheck.CommentCount != checkCompare.CommentCount)                    {                        var msg = "At " + DateTime.Now.ToShortTimeString() + " Found "                             + (currentCheck.CommentCount - checkCompare.CommentCount)                             + " New Comment(s) in thread, " + checkCompare.Title;<br>                        listBoxLog.Items.Add(msg);<br>                        sb.AppendLine(msg);<br>                        newComment = <span class="cpp-keyword">true</span>;<br>                        notifyIcon1.Text = <span class="cpp-literal">"New Comment(s) found"</span>;<br>                    }<br>                }<br>            }<br><br><br><br><br><br></pre></div><!–ENDSCRIPT–><br><br>You can also save the last query results to disk. Then later on you may download from the site and check against disk so the program doesnt have to run continuosly to be useful.<br><br><!–STARTSCRIPT–><!–source lang="cpp"–><div class="source"><pre>     <span class="cpp-keyword">private</span> <span class="cpp-keyword">void</span> buttonSaveState_Click(object sender, EventArgs e)<br>        {<br>            var sb = <span class="cpp-keyword">new</span> StringBuilder();<br>            oldCheck.ForEach(entry => sb.AppendLine (entry.CommentCount + <span class="cpp-literal">"|"</span>+entry.IsComment + <span class="cpp-literal">"|"</span> +entry.Title));<br>            File.WriteAllText(statePath, sb.ToString());            <br>        }<br><br>        <span class="cpp-keyword">private</span> <span class="cpp-keyword">void</span> buttonCompareStates_Click(object sender, EventArgs e)<br>        {<br>            var urlFromFile = System.IO.File.ReadAllText(infFilePath);<br>            <span class="cpp-keyword">if</span> ((urlFromFile != textBoxUrl.Text)  && textBoxUrl.Text != <span class="cpp-literal">""</span>)<br>                MessageBox.Show(<span class="cpp-literal">"Warning Url found in file and in textbox do not match. This *may* cause discrapncies."</span>,<br>                                  <span class="cpp-literal">"Are you Sure you know what you are doing?"</span>,<br>                                  MessageBoxButtons.OK, MessageBoxIcon.Warning);<br>            var tmpCheck = <span class="cpp-keyword">new</span> List<JournalEntry>();<br>            var dat = File.ReadAllLines(statePath).ToList ();<br>            dat.ForEach(item => {   var s = item.Split('|');<br>                                    var newE = <span class="cpp-keyword">new</span> JournalEntry();<br>                                    newE.CommentCount = <span class="cpp-keyword">int</span>.Parse(s[<span class="cpp-number">0</span>]);<br>                                    newE.IsComment = <span class="cpp-keyword">bool</span>.Parse(s[<span class="cpp-number">1</span>]);<br>                                    newE.Title = s[<span class="cpp-number">2</span>];                <br>                                    tmpCheck.Add(newE); });<br>            var currentCheck = util.PollGDNEt(CheckUrlOptions());<br>            inited = <span class="cpp-keyword">true</span>;<br>            DoComparison(currentCheck, tmpCheck);<br>        }<br><br><br><br><br><br></pre></div><!–ENDSCRIPT–><br><br>The part that scrapes the page is here <br><br><!–STARTSCRIPT–><!–source lang="cpp"–><div class="source"><pre><span class="cpp-keyword">public</span> <span class="cpp-keyword">static</span> List<JournalEntry> PollGDNEt(string url)<br>        {<br>            HtmlWeb page = <span class="cpp-keyword">new</span> HtmlWeb();                        <br>            HtmlDocument doc = page.Load(url);            <br>            var xdoc = doc.ToXDocument();<br><br>            var queryResults = from element in xdoc.Descendants()<br>                    where element.HasAttributes  <br>                    && element.Name.LocalName == <span class="cpp-literal">"span"</span><br>                    && (element.FirstAttribute.Value == <span class="cpp-literal">"regularfont"</span> || (element.FirstAttribute.Value == <span class="cpp-literal">"smallfont"</span> && element.Value.Contains(<span class="cpp-literal">"Comments"</span>)))<br>                    select <span class="cpp-keyword">new</span> {Title = element.Value, <br>                                IsComment = element.FirstAttribute.Value == <span class="cpp-literal">"smallfont"</span>, <br>                                Count = element.FirstAttribute.Value == <span class="cpp-literal">"smallfont"</span> ? <br>                                       <span class="cpp-keyword">int</span>.Parse( Regex.Match( element.Value, @<span class="cpp-literal">"\d+"</span>).Value  ) : <span class="cpp-number">0</span>  };<br>           var Entries = <span class="cpp-keyword">new</span> List<JournalEntry>();<br>           foreach (var result in queryResults)<br>           {<br>               var entry = <span class="cpp-keyword">new</span> JournalEntry (); <br>               entry.CommentCount = result.Count; entry.IsComment = result.IsComment ; entry.Title = result.Title ;<br>               Entries.Add(entry);<br>           }<br>           <span class="cpp-keyword">return</span> Entries;<br>        }<br><br><br><br><br><br></pre></div><!–ENDSCRIPT–><br><br>Full Source is <a href = "http://svn.xp-dev.com/svn/dynamic_jester_Jargon_Flies_Repository2/gdjournal_comments/GameDevJournalCommentator/">here (svn)</a> or <a href = "http://svn.xp-dev.com/svn/dynamic_jester_Jargon_Flies_Repository2/gdjournal_comments/GameDevJournalCommentator.rar">source rar'd</a>. And app is <a href = "http://svn.xp-dev.com/svn/dynamic_jester_Jargon_Flies_Repository2/gdjournal_comments/GameDevJournalCommentator/bin/Debug/gdevj.rar">here (requires .NET 3.5)</a>. Some stats in order of # comments found on page:<br><pre><b><br>User       |  Total comments on Page   |   Avg Comments Per Post  |   Total Posts</b><br>——————————————————————————————————-<br>TrapperZoid      20                                       4                                 5<br>Drew             19                                       1.27                             15<br>Talestyn         16                                       1.07                             15<br>Ben              16                                       2                                 8<br>Ravuya           16                                       1.14                            14<br>Me               6                                        1.2                              5<br></pre><br><hr> Also Mike P if I you read this and I could get the source for your line counter project, would be cool.<div>


</div>
Previous Entry RAE 2008 + Random
0 likes 13 comments

Comments

Daerax
tst
December 20, 2008 06:33 PM
Daerax
tst
December 20, 2008 07:04 PM
Daerax
t
December 20, 2008 07:51 PM
Mike.Popoloski
Quote:Also Mike P if I you read this and I could get the source for your line counter project, would be cool.


Sure thing, although you could have just PMed me :D

Actually, the source is online and downloadable via SVN, although if you're extremely lazy, you could ask me nicely and I might zip up a copy for you.

The code isn't all that pretty, and I make no promises. Right now it actually tabulates totals for all files as well as per-filter, but I have no good way of showing that right now (HtmlReport isn't finished yet), so I just print the grand totals to the console. If you fix it up, be sure to let me know.
December 20, 2008 10:24 PM
nerd_boy
That, sir, is an excellent idea. I've downloaded it, and only messed with it a little bit, but it seems *quite nice*. Might have to mess with it and see what else could be done with it.

Also, cookie++
December 21, 2008 04:19 AM
Daerax
Quote:Original post by nerd_boy
That, sir, is an excellent idea. I've downloaded it, and only messed with it a little bit, but it seems *quite nice*. Might have to mess with it and see what else could be done with it.

Also, cookie++


Thanks :)
December 21, 2008 04:02 PM
Daerax
Quote:Original post by Mike.Popoloski
Quote:Also Mike P if I you read this and I could get the source for your line counter project, would be cool.


Sure thing, although you could have just PMed me :D

Actually, the source is online and downloadable via SVN, although if you're extremely lazy, you could ask me nicely and I might zip up a copy for you.

The code isn't all that pretty, and I make no promises. Right now it actually tabulates totals for all files as well as per-filter, but I have no good way of showing that right now (HtmlReport isn't finished yet), so I just print the grand totals to the console. If you fix it up, be sure to let me know.


I downloaded via svn so wasnt much effort so we both got to be lazy :). Firstly thanks alot Ive been looking for a line counter without all the unneccessary bells and whistles or exorbitant pricing.

So Ive gotten the program to a shape I am happy with. The config.txt was not tied down to anyone location so was saving in some random dir with no hope of my finding it. I tied it down to same dir as app. I also added a summary field to FileNode which displays all the info per file in the console and spits out to a text and csv file so nice graphs can be made. I also added optional removal of counting for things which cause bloat in the code via regular expressions contained in a filter_config.txt (for example I dont want *.designer.cs to be counted so i have a ^.*\.Designer.*$|^AssemblyInfo\..*$ filter). Finally I added the ability to ignore project files and simply recursively count all files in a folder, with counting of code per file type with percentages as well - (built on top of and not breaking your layout :) - which is useful for asp.net web sites. Necessary changes to command line were done (linecounter folderpath [/fr |/f] for example). The rar'd code changes can be found here for your review.

Sample Output
Total Lines : 749
Code Lines  : 621
Comments    : 2
Blank Lines : 126

------------------------

Tables.fs : Blank Lines: 13,  Code Lines: 39, Comments: 0, Total Lines: 52
------------------------


------------------------

VCProject.cs : Blank Lines: 11,  Code Lines: 43, Comments: 0, Total Lines: 54
------------------------


------------------------

Program.cs : Blank Lines: 13,  Code Lines: 95, Comments: 0, Total Lines: 108
------------------------


------------------------

Nodes.cs : Blank Lines: 3,  Code Lines: 23, Comments: 0, Total Lines: 26
------------------------


------------------------

HtmlReport.cs : Blank Lines: 17,  Code Lines: 64, Comments: 0, Total Lines: 81
------------------------


------------------------

Helpers.cs : Blank Lines: 6,  Code Lines: 26, Comments: 0, Total Lines: 32
------------------------


------------------------

CSProject.cs : Blank Lines: 11,  Code Lines: 66, Comments: 0, Total Lines: 77
------------------------


------------------------

Counter.cs : Blank Lines: 52,  Code Lines: 265, Comments: 2, Total Lines: 319
------------------------

.cs files, Count 7, Total Lines 697, Code Lines 582 [Percentage: 94 %]

.fs files, Count 1, Total Lines 52, Code Lines 39 [Percentage: 6 %]



December 21, 2008 05:23 PM
nerd_boy
>_> How the flippin' hoo-haa did you get your post between mine and Mike.Popoloski's!? [disturbed]
December 22, 2008 03:06 AM
Evil Steve
Quote:Original post by nerd_boy
>_> How the flippin' hoo-haa did you get your post between mine and Mike.Popoloski's!? [disturbed]
Like this [smile]
December 22, 2008 04:05 AM
Mike.Popoloski
Quote:Original post by Daerax*snip*


Cool, that's the sort of thing I wanted to add, but simply didn't have time for. I still have a few other ideas, but for now it's one of the lowest priorities in my ever-growing list of projects I want to work on.

Anyways, glad it could be of some use for someone.
December 22, 2008 09:11 AM
benryves
Great idea for a utility! [grin] Cheers!
December 22, 2008 03:58 PM
Daerax
Quote:Original post by benryves
Great idea for a utility! [grin] Cheers!


Thanks hah. I got a bubble stating you posted earlier heh.
December 22, 2008 04:23 PM
Daerax
tst2
December 22, 2008 08:10 PM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Profile
Author
Advertisement
Advertisement