Jump to content

  • Log In with Google      Sign In   
  • Create Account

#ActualHodgman

Posted 20 November 2012 - 08:30 AM

1) If you want to block until it's loaded, then why use async loading? The point of async loading is that you can display a nice "please wait" animation instead of blocking, and then receive a notification when your asset is ready to stop the animation and start using the asset.

2) Yeah, often the function that initiates a request returns some kind of identifier that you can use to check on it's status or abort the request. You probably don't need to build timeouts into the system at this level, as long as you've got the ability to abort at this level (even that's not required for many games Posted Image) Then, the systems that use the file loader can implement user-triggered, or time-based, etc aborting, if they require it.

3) With decompression, it's common for the decompression thread to keep it's own buffers for re-use.

If you are reading a lot of files into temporary memory before parsing them and throwing the allocations away, it might make sense to allocate a small ring of buffers for the file loader to use as defaults. You'd have to collect some statistics on the size of your "temporary" files and how many are being loaded at once to make an educated guess as to the best sizes to use.

As for "allocate buffer, async read into buffer, instantiate resource from buffer, free buffer", you can often just "instantiate resource, async read into resource". I've split my async asset loading into a few steps to help with this --
When you issue an async request, you provide a callback for allocating the asset, and a callback for parsing the asset. When the file size becomes known (I store a table of sizes as part of the file system, which is resident in RAM, so this is immediate), the allocation callback is triggered to create the resource. Then the resource is filled in asynchronously, and when it's done the 'parsing' callback is triggered to finalize the resource. Assets can also report an array of sizes if they require some of their data to be streamed into different allocations (e.g. a texture header for the CPU and the actual texel data for the GPU).

4) If you want to write OS-specific code, you can usually implement async file loading without any extra threads, as OS file interations are likely async natively (the blocking API is often a wrapper around the async API). However, if you want to do background decompression, that obviously does require some threading ;)

#4Hodgman

Posted 20 November 2012 - 08:29 AM

1) If you want to block until it's loaded, then why use async loading? The point of async loading is that you can display a nice "please wait" animation instead of blocking, and then receive a notification when your asset is ready to stop the animation and start using the asset.

2) Yeah, often the function that initiates a request returns some kind of identifier that you can use to check on it's status or abort the request. You probably don't need to build timeouts into the system at this level, as long as you've got the ability to abort at this level (even that's not required for many games Posted Image) Then, the systems that use the file loader can implement user-triggered, or time-based, etc aborting, if they require it.

3) With decompression, it's common for the decompression thread to keep it's own buffers for re-use.

If you are reading a lot of files into temporary memory before parsing them and throwing the allocations away, it might make sense to allocate a small ring of buffers for the file loader to use as defaults. You'd have to collect some statistics on the size of your "temporary" files and how many are being loaded at once to make an educated guess as to the best sizes to use.

As for "allocate buffer, async read into buffer, instantiate resource from buffer, free buffer", you can often just "instantiate resource, async read into resource". I've split my async asset loading into a few steps to help with this --
When you issue an async request, you provide a callback for allocating the asset, and a callback for parsing the asset. When the file size becomes known, the allocation callback is triggered to create the resource. Then the resource is filled in asynchronously, and when it's done the 'parsing' callback is triggered to finalize the resource. Assets can also report an array of sizes if they require some of their data to be streamed into different allocations (e.g. a texture header for the CPU and the actual texel data for the GPU).

4) If you want to write OS-specific code, you can usually implement async file loading without any extra threads, as OS file interations are likely async natively (the blocking API is often a wrapper around the async API). However, if you want to do background decompression, that obviously does require some threading ;)

#3Hodgman

Posted 20 November 2012 - 08:27 AM

1) If you want to block until it's loaded, then why use async loading? The point of async loading is that you can display a nice "please wait" animation instead of blocking, and then receive a notification when your asset is ready to stop the animation and start using the asset.

2) Yeah, often the function that initiates a request returns some kind of identifier that you can use to check on it's status or abort the request. You probably don't need to build timeouts into the system at this level, as long as you've got the ability to abort at this level (even that's not required for many games Posted Image) Then, the systems that use the file loader can implement user-triggered, or time-based, etc aborting, if they require it.

3) With decompression, it's common for the decompression thread to keep it's own buffers for re-use.

As for "allocate buffer, async read into buffer, instantiate resource from buffer, free buffer", you can often just "instantiate resource, async read into resource". I've split my async asset loading into a few steps to help with this --
When you issue an async request, you provide a callback for allocating the asset, and a callback for parsing the asset. When the file size becomes known, the allocation callback is triggered to create the resource. Then the resource is filled in asynchronously, and when it's done the 'parsing' callback is triggered to finalize the resource. Assets can also report an array of sizes if they require some of their data to be streamed into different allocations (e.g. a texture header for the CPU and the actual texel data for the GPU).

4) If you want to write OS-specific code, you can usually implement async file loading without any extra threads, as OS file interations are likely async natively (the blocking API is often a wrapper around the async API). However, if you want to do background decompression, that obviously does require some threading ;)

#2Hodgman

Posted 20 November 2012 - 08:22 AM

1) If you want to block until it's loaded, then why use async loading? The point of async loading is that you can display a nice "please wait" animation instead of blocking, and then receive a notification when your asset is ready to stop the animation and start using the asset.

2) Yeah, often the function that initiates a request returns some kind of identify that you can use to check on it's status or abort the request. You probably don't need to build timeouts into the system at this level, as long as you've got the ability to abort at this level. Then, the systems that use the file loader can implement user-triggered, or time-based, etc aborting, if they require it.

3) With decompression, it's common for the decompression thread to keep it's own buffers for re-use.

As for "allocate buffer, async read into buffer, instantiate resource from buffer, free buffer", you can often just "instantiate resource, async read into resource". I've split my async asset loading into a few steps to help with this --
When you issue an async request, you provide a callback for allocating the asset, and a callback for parsing the asset. When the file size becomes known, the allocation callback is triggered to create the resource. Then the resource is filled in asynchronously, and when it's done the 'parsing' callback is triggered to finalize the resource. Assets can also report an array of sizes if they require some of their data to be streamed into different allocations (e.g. a texture header for the CPU and the actual texel data for the GPU).

4) If you want to write OS-specific code, you can usually implement async file loading without any extra threads, as OS file interations are likely async natively (the blocking API is often a wrapper around the async API). However, if you want to do background decompression, that obviously does require some threading ;)

#1Hodgman

Posted 20 November 2012 - 08:21 AM

1) If you want to block until it's loaded, then why use async loading? The point of async loading is that you can display a nice "please wait" animation instead of blocking, and then receive a notification when your asset is ready to stop the animation and start using the asset.

2) Yeah, often the function that initiates a request returns some kind of identify that you can use to check on it's status or abort the request. You probably don't need to build timeouts into the system at this level, as long as you've got the ability to abort at this level. Then, the systems that use the file loader can implement user-triggered, or time-based, etc aborting, if they require it.

3) With decompression, it's common for the decompression thread to keep it's own buffers for re-use.

As for "allocate buffer, async read into buffer, instantiate resource from buffer, free buffer", you can often just "instantiate resource, async read into resource". I've split my async asset loading into a few steps to help with this --
When you issue an async request, you provide a callback for allocating the asset, and a callback for parsing the asset. When the file size becomes known, the allocation callback is triggered to create the resource. Then the resource is filled in asynchronously, and when it's done the 'parsing' callback is triggered to finalize the resource.

4) If you want to write OS-specific code, you can usually implement async file loading without any extra threads, as OS file interations are likely async natively (the blocking API is often a wrapper around the async API). However, if you want to do background decompression, that obviously does require some threading ;)

PARTNERS