Sign in to follow this  
cowboygun

Want a 1ms tick from Linux without modifying Linux Kernel

Recommended Posts


I need to develop an embedded application running on Linux, to be more specific, Wind River Linux 4.0 is my favorite for now. My application requires a timer, or scheduler, which is required to tick my application for about every 1ms. I say "about" because there is no strict timing requirements, and either 1.01ms or 0.98ms would be acceptable. In my application, written in C++, I will implement OBSERVER pattern and Listener paradigm to get a full-featured scheduler to dispatch events to processes at variable rates, say 5ms, 10ms, etc.

I would like to know if I can accomplish the above design without modifying the linux kernel, since under GPL, having a kernel module in my application will cause my application to be GPLed. At this moment I don't want to go that far yet. If there is something in the kernel already available, and can tick my application every 1ms, I would like to use it directly in my application. I hope this way will save me from the GPL license issue. Any thoughts? Thank you in advance!

Share this post


Link to post
Share on other sites
My out-of-the-box Debian and Ubuntu Linux systems have timeout on epoll_wait (and thus probably on everything else too, but I didn't test that explicitly) working at accurracies in the microsecond (micro, not milli) range without having to refer to any special RT stuff.

So, a millisecond should be no issue at all (assuming every "normal" Linux works more or less the same).

Share this post


Link to post
Share on other sites
Quote:
Original post by samoth
My out-of-the-box Debian and Ubuntu Linux systems have timeout on epoll_wait (and thus probably on everything else too, but I didn't test that explicitly) working at accurracies in the microsecond (micro, not milli) range without having to refer to any special RT stuff.

So, a millisecond should be no issue at all (assuming every "normal" Linux works more or less the same).


What is the *granularity* of epoll_wait? If you call epoll_wait 100μs apart, will it wait for 100μs or will it wait somewhere between 0 and 20ms (like Windows do)?

Share this post


Link to post
Share on other sites
The former.

I tested that once when I had the exact same question (how to block somewhat reliably for 1ms) as the OP, and found that this is really nothing to worry about.

While I don't remember whether I ran the tests over 1 second, or 10 seconds, 100 seconds, or whatever time, the baseline was that if you ask it to block for 1μs a million times, the total time will add up to slightly over a second (as opposed to 2000 seconds). So, in wall time, you get pretty much exactly what you asked for, plus some syscall overhead. Which honestly, I found absolutely stunning.

Share this post


Link to post
Share on other sites
Using epoll() is overkill when you could just use timerfd_create() can friends for a nanosecond-level timer. Of course, you can multiplex a timerfd through epoll(), poll(), select(), or do a blocking read() in a thread (whatever your favorite multiplexing poison is), but using timerfd you can have multiple timers pending and leave the dispatch to the kernel, yielding simpler code and lower system overhead.

Share this post


Link to post
Share on other sites
Quote:
Original post by Palidine
http://linux.die.net/man/3/clock_gettime

use clock_getres clock_gettime with CLOCK_REALTIME like you could QueryPerformanceCounter/QueryPerformanceFrequency on Windows

-me


It is a solution but I don't want to query the timer in my application. I actually need a soft interrupt or event.

Share this post


Link to post
Share on other sites
Quote:
Original post by Sirisian
Linux can do pretty good ms resolution events. I recommend just using Boost Deadline Timer. After googling for boost periodic timer this came up if you want a periodic timer.


This sounds a good solution. Is there any license issue with this ASIO timer? It seems some libraries may be involved.

#include <iostream>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>

Share this post


Link to post
Share on other sites
Quote:
Original post by Steve132
Also, the OpenMP timer has absurdly small resolution and extremely high accuracy (also thread-safe!)


Is it an easy job to add OpenMP timer to an existing application? We don't actually need parallel computation.

Share this post


Link to post
Share on other sites
Quote:
Original post by samoth
My out-of-the-box Debian and Ubuntu Linux systems have timeout on epoll_wait (and thus probably on everything else too, but I didn't test that explicitly) working at accurracies in the microsecond (micro, not milli) range without having to refer to any special RT stuff.

So, a millisecond should be no issue at all (assuming every "normal" Linux works more or less the same).


It sounds good to my application. I will check if it is available at Wind River Linux 4.

Share this post


Link to post
Share on other sites
Quote:
Original post by Bregma
Using epoll() is overkill when you could just use timerfd_create() can friends for a nanosecond-level timer. Of course, you can multiplex a timerfd through epoll(), poll(), select(), or do a blocking read() in a thread (whatever your favorite multiplexing poison is), but using timerfd you can have multiple timers pending and leave the dispatch to the kernel, yielding simpler code and lower system overhead.


It is a solution but I don't want to query the timer in my application. I actually need a soft interrupt or event.

Share this post


Link to post
Share on other sites
Quote:
Original post by cowboygun
Quote:
Original post by Sneftel
Google "RT-PREEMPT".

Is it an overkill to turn the Linux into a fully preemptible kernel?
If you're only really running the one process, then yes. This patch is useful if you'll have several processes, of different priorities, and want to avoid any nasty dependency inversion surprises. I mostly mention it, though, because the page which talks about it has a lot of good information on doing real-time computing under linux.

Share this post


Link to post
Share on other sites
Quote:
Original post by cowboygun
Quote:
Original post by Steve132
Also, the OpenMP timer has absurdly small resolution and extremely high accuracy (also thread-safe!)


Is it an easy job to add OpenMP timer to an existing application? We don't actually need parallel computation.


Yes. The timing functions are a part of the libomp implementation, which is implemented by the compiler vendors. You don't even have to link, you just have to enable openmp support in the compiler and #include<omp.h>

However, the parallel computation features are very easy to use as well.

Share this post


Link to post
Share on other sites
The problem with using user-space solutions such as epoll/select/timerfd etc is that while they will work, they're not actually a "real-time" solution in that service is not guaranteed.

Your process *could* be suspended for (essentially) arbitrary amounts of time depending on what other things are happening in the system. For example; inside those system calls, the kernel may go do dirty page flushes for other processes. And it may wait on those page writes completing before returning.

This is the essential difference between "real-time" and "multi-tasking" systems -- a real-time environment guarantees a service delivery window.

Depending on what you're doing, this may or may not be acceptable, but you need to know that it's happening.

Apart from that, the advice to use multiple timerfds and run an epoll gather/dispatch loop is probably the right way to go. It's fairly easy to build a service object which creates a timerfd and hooks it into the epoll set storing a this pointer into the epoll userdata. Inherit from this implementing your service method. The main loop calls the epoll, gathers the event and just calls the service vmethod through the userdata pointer.

There is a single "gotcha" with this -- if you delete an object, it may already have been signalled and hence stored into the event queue. So you need to keep an eye out for stale pointers. Apart from that, the epoll sets are all thread-safe (because they're kernel internals) so you don't need to worry about that stuff.

Share this post


Link to post
Share on other sites
Quote:
Original post by Steve132
Quote:
Original post by cowboygun
Quote:
Original post by Steve132
Also, the OpenMP timer has absurdly small resolution and extremely high accuracy (also thread-safe!)


Is it an easy job to add OpenMP timer to an existing application? We don't actually need parallel computation.


Yes. The timing functions are a part of the libomp implementation, which is implemented by the compiler vendors. You don't even have to link, you just have to enable openmp support in the compiler and #include<omp.h>

However, the parallel computation features are very easy to use as well.


Interesting! I will give it a try.

Thank you for sharing!

Share this post


Link to post
Share on other sites
Quote:
Original post by Katie
The problem with using user-space solutions such as epoll/select/timerfd etc is that while they will work, they're not actually a "real-time" solution in that service is not guaranteed.

Your process *could* be suspended for (essentially) arbitrary amounts of time depending on what other things are happening in the system. For example; inside those system calls, the kernel may go do dirty page flushes for other processes. And it may wait on those page writes completing before returning.

This is the essential difference between "real-time" and "multi-tasking" systems -- a real-time environment guarantees a service delivery window.

Depending on what you're doing, this may or may not be acceptable, but you need to know that it's happening.

Apart from that, the advice to use multiple timerfds and run an epoll gather/dispatch loop is probably the right way to go. It's fairly easy to build a service object which creates a timerfd and hooks it into the epoll set storing a this pointer into the epoll userdata. Inherit from this implementing your service method. The main loop calls the epoll, gathers the event and just calls the service vmethod through the userdata pointer.

There is a single "gotcha" with this -- if you delete an object, it may already have been signalled and hence stored into the event queue. So you need to keep an eye out for stale pointers. Apart from that, the epoll sets are all thread-safe (because they're kernel internals) so you don't need to worry about that stuff.


Very useful information. I am actually thinking about ordering a customerized CSP (chip support package) from Wind River so I can get a "real-time" kernel instead of "multi-tasking" one. I think I should add the following into the requirement of CSP: A easy-to-use API to start a timer and get software interrupt or event; The kernel should be real-time 1000Hz one which means it can generate the software interrupt or event for at least every 1ms. Does the requirements make sense?

I am also considering epoll/epoll_wait/select/timerfd, it may become an elegant solution too, since I think my application doesn't have strict real-time requirement. I need some tests though. Thank you for pointing out the "gotcha'.







Share this post


Link to post
Share on other sites
Quote:
Original post by cowboygun
Quote:
Original post by Sirisian
Linux can do pretty good ms resolution events. I recommend just using Boost Deadline Timer. After googling for boost periodic timer this came up if you want a periodic timer.


This sounds a good solution. Is there any license issue with this ASIO timer? It seems some libraries may be involved.

License issue? It's just the boost license. I don't want to jump to conclusion, but if you've never used boost I recommend checking it out. It has amazing wrappers around threads and pointer management.

Share this post


Link to post
Share on other sites
Quote:
Original post by cowboygun

Very useful information. I am actually thinking about ordering a customerized CSP (chip support package) from Wind River so I can get a "real-time" kernel instead of "multi-tasking" one. I think I should add the following into the requirement of CSP: A easy-to-use API to start a timer and get software interrupt or event; The kernel should be real-time 1000Hz one which means it can generate the software interrupt or event for at least every 1ms. Does the requirements make sense?

Hard real-time guarantees that an event will be generated by a certain deadline, or the system faults. Resolution itself is not part of that. But if something claims 1ms resolution, then that is the smallest resolution at which events can be scheduled.

With hard real time, things can get complicated. If something needs to be done in response to event, it may take too long and cause other events to be delayed and system fails as a consequence.

What is this for? Hard real-time programming is frequently much different from conventional one due to all these constraints.

Quote:
Linux can do pretty good ms resolution events. I recommend just using Boost Deadline Timer. After googling for boost periodic timer this came up if you want a periodic timer.
Last time I checked, asio used O(n) event dispatching which meant it broke down after some 200 scheduled events. It's definitely not a real-time scheduler.

Share this post


Link to post
Share on other sites
Quote:
Original post by samoth
My out-of-the-box Debian and Ubuntu Linux systems have timeout on epoll_wait (and thus probably on everything else too, but I didn't test that explicitly) working at accurracies in the microsecond (micro, not milli) range without having to refer to any special RT stuff.

So, a millisecond should be no issue at all (assuming every "normal" Linux works more or less the same).


Is epoll or epoll_wait still a method based on polling? I actually need an interrupt-like timer.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this