I want to have one thread enum files in user folder and possible recurse them. and while one thread is doing that i want another checking the file enum data to see if it should be filter in or out. What would be the best way to do this? I have no idea how quick the filters will be. The file can fail (or pass) on the first check or it can go through dozens of checks and fail on the last check. So i may be able to enum 5 files before the first check is done, then do checks 3 files before 1 file enum is done. How should i manage these threads? the enum thread ends when all files has been enum'd and then the filter flag will end when all files enum data has been checked. then the main thread will use the filter files how it pleases. I am mostly worried about locking/unlocking hundreds of thousands of times. I was considering having a volatile ulong fileEnumCount, isFinished; have the filter file check for the two variable and keep doing sleep(0) until isFinished is true and enumScanIndex == fileEnumCount. But that feels to much of a hack

Actually i am not testing for a enum value, i am enuming the file data and checking its size, filename, time access, etc. then i am testing it though either simple or very complex filters.

I know the HD is slow, but i dont want to scan 1 file, run every test then scan another file. i rather keep the hd enum going so i dont have to wait.

I had the feeling inter-thread communication is expensive. Right now that volatile variable looks like a good solution. Anyone else have any comments about how i should do this?

Quote:Original post by AcidZombie24
I had the feeling inter-thread communication is expensive. Right now that volatile variable looks like a good solution.

They're not a solution. Strictly speaking, the volatile keyword is pretty much useless for multi-threading. Newer versions of MSVC give you certain guarantees, but I would feel very uncomfortable relying on them.

Quote:Original post by AcidZombie24
I know the HD is slow, but i dont want to scan 1 file, run every test then scan another file.

Why not ? As Antheus said, multithreading this will gain you absolutely nothing here. It will probably slow everything down significantly.

Unless your filters are ridiculously expensive. But then you're doing something very wrong somewhere.

Quote:Original post by AcidZombie24
I had the feeling inter-thread communication is expensive. Right now that volatile variable looks like a good solution. Anyone else have any comments about how i should do this?

Your current 'solution' is just asking for race conditions or deadlocks. You'll need to properly secure them with mutexes, condition variables, or something similar.

Quote:Original post by Yann L
Why not ? As Antheus said, multithreading this will gain you absolutely nothing here. It will probably slow everything down significantly.

Unless your filters are ridiculously expensive. But then you're doing something very wrong somewhere.

even with a volatile variable solution? i am still thinking this is better. I may paste the code up to see what you guys think. I am sure i can do this w/o taking a significant thread hit.

Quote:Original post by Yann L

Quote:Original post by AcidZombie24
I had the feeling inter-thread communication is expensive. Right now that volatile variable looks like a good solution. Anyone else have any comments about how i should do this?

Your current 'solution' is just asking for race conditions or deadlocks. You'll need to properly secure them with mutexes, condition variables, or something similar.

What is so unsecured with a volatile? only 1 thread is updating it (increase only) and the other thread is only reading it to see if there is more data to process.

Quote:Original post by AcidZombie24
even with a volatile variable solution?

The volatile modifier has absolutely nothing to do with it. Unless you are doing very time consuming operations on the listed files (eg. parsing them or similar), multithreading is a poor choice for this task.

Give us an example of the most complex filter you have.

Quote:Original post by AcidZombie24
What is so unsecured with a volatile? only 1 thread is updating it (increase only) and the other thread is only reading it to see if there is more data to process.

Volatile is not a synchronisation primitive. And if you really want to multithread this process, then you'll need a lot more than just a one variable single-way communication channel.

come to think of it. the enum will destory the data so i would have to copy the struct if i wanted to do that, which i dont >_<.

btw, all i would need is 2, isFinished, fileCount. i check proccessedFileCount against fileCount, when == i check if it finished. if so, it ends. otherwise it sleeps and waits.

*kicks self*