Jump to content
  • Advertisement
Sign in to follow this  
  • entries
  • comments
  • views

Syntax hilighting

Sign in to follow this  


So, something I've been working on a fair amount recently is a new syntax hilighter for GDNet. The existing hilighter supports something like four languages, runs in VBScript, and is basically just a small number of regexp passes over the code.

What I'm looking at doing is replacing it with a component that I've built around the colorer library. Colorer supports over 100 languages, has somewhat more powerful parsing capabilities, and is written in native C++ such that the component I've built around it can process a 3K file of Python source in ~500ms, 1K of C++ in ~90ms, and 95K of C++ in ~3800ms. There's still more optimization to be done.

Here's a couple of examples of the kind of output I'm producing: 1, 2. Take a look at the HTML sources to those. There's a lot of detail in there.

Some of the languages that colorer supports are missing potential extensions - Oluseyi suggested hilighting standard library functions in Python, for example - but it's open-source, and the way you specify a language for the parser is a published standard, so it shouldn't be too hard for people to do the bulk of my work for me on those things. We'll also benefit from any new languages that other people using Colorer contribute, or optimizations they make to the parsing engine, while also keeping our ability to develop it further ourselves if we want to.

This is how OSS is supposed to work - it just frequently doesn't. (I'm still to discover exactly how I contribute the changes I've made to the code back into the codebase on sourceforge).
Sign in to follow this  


Recommended Comments

Cool. I didn't even know that an improved syntax highlighter was on the radar for enhancements. Is there a to-do list of gamedev improvements and bugfixes somewhere?

Share this comment

Link to comment
Well, you are able to view things in my JIRA project - but truth be told, this is really a V5 component. Every part of the site that currently happens in ASP (which is pretty much all of it) has to get pushed out to .NET or COM or some other replacement that can be used from ASP.NET. The syntax hilighter just happens to be one of the parts that is neatly self-contained such that changing V4 of the site to use the V5 component isn't a massive undertaking.

So, consider it like this: Every part of the site is on the radar for replacement, and if it's sensible to actually upgrade a part while it's being replaced, then we'll do it.

Share this comment

Link to comment
Good work - is it the 3K Python file that is now your 30k HTML file? [lol]

Any chance of getting HLSL/GLSL/Cg/FX syntax highlighting? Even Visual Studio can't hack that!


PS - you raise an interesting point about OSS and how hard it is to contribute. I've hit that before.

Share this comment

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!