• Advertisement
  • Popular Tags

  • Popular Now

  • Advertisement
  • Similar Content

    • By Ty Typhoon
      ...if you got time to read and answer i would be happy . 
       
      So me and my co try to do a game.
      It should be in unity couse my co do everything in this engine.
       
      We got the rpg package from evila for inventor, but it only runs on pc right now.
       
      I like to make a online store for guns in the game and a multiplayer open world that runs on pc, android, mobile, ps4, xbox one. 
      Somebody told me that you "only" need to program it like so and that its possible in every engine...
       
      So if you are one of the lucky guys who could help me out or programm that, or even if you know a newer better package for maybe unreal which offers that - please let me know now.
    • By Ty Typhoon
      I like to build my A - Team now.
       
      I need loyal people who can trust and believe in a dream.
      If you got time and no need for direct pay please contact me now.
       
      We cant pay now, you will recieve a lifetime percentage if the released game will give earnings. 
      If I get the best people together for a team, everything should be possible.
       
       
      What i need:
      - Programmer c++
      - Unity / Unreal - we must check whats possible, please share your experience with me.
      - Sculpter, 3D Artist
      - Animator
      - Marketing / Promotion 
       
       
      What i do:
      - Studio Owner
      - Director
      - Recruit exactly you
      - Sounddesign
      - Main theme composing
      - Vocals
      - Game design
      - Gun, swords, shields and weapon design
      - Character, plants and animal design
       
       
      Please dont ask about the Name of the Game, about Designs or Screenshots.
      The game will be defintitly affected about our and your skills if you join the team.
       
       
      Planned for the big Game:
      - 1st person shooter
      - online multiplayer
      - character manipulation
      - complete big open world with like lifetime actions and reactions
      - gunstore with many items to buy
      - many upgrades for players
      - specials like mini games
       
      So if you are interested in joining a team with a nearly complete game idea, contact me now and tell me what you can do.
       
      discord:
      joerg federmann composing#2898
       
       
    • By codelyoko373
      I wasn't sure if this would be the right place for a topic like this so sorry if it isn't.
      I'm currently working on a project for Uni using FreeGLUT to make a simple solar system simulation. I've got to the point where I've implemented all the planets and have used a Scene Graph to link them all together. The issue I'm having with now though is basically the planets and moons orbit correctly at their own orbit speeds.
      I'm not really experienced with using matrices for stuff like this so It's likely why I can't figure out how exactly to get it working. This is where I'm applying the transformation matrices, as well as pushing and popping them. This is within the Render function that every planet including the sun and moons will have and run.
      if (tag != "Sun") { glRotatef(orbitAngle, orbitRotation.X, orbitRotation.Y, orbitRotation.Z); } glPushMatrix(); glTranslatef(position.X, position.Y, position.Z); glRotatef(rotationAngle, rotation.X, rotation.Y, rotation.Z); glScalef(scale.X, scale.Y, scale.Z); glDrawElements(GL_TRIANGLES, mesh->indiceCount, GL_UNSIGNED_SHORT, mesh->indices); if (tag != "Sun") { glPopMatrix(); } The "If(tag != "Sun")" parts are my attempts are getting the planets to orbit correctly though it likely isn't the way I'm meant to be doing it. So I was wondering if someone would be able to help me? As I really don't have an idea on what I would do to get it working. Using the if statement is truthfully the closest I've got to it working but there are still weird effects like the planets orbiting faster then they should depending on the number of planets actually be updated/rendered.
    • By BenjaminBouchet
      Learning game development in Unreal Engine could be a daunting task for someone who don’t know where to start, and a cumbersome process if you don’t organize your progression correctly. One thing commonly known by experienced developers and by people unfamiliar with coding: mastering a development language is a long and difficult task.
      From blueprints to C++ in Unreal Engine
      If you want to learn fast, you need a good learning strategy. Unreal Engine contains a very powerful tool which you can use to learn C++ faster: its blueprint system. Blueprints are extremely easy to learn (and you may already have a good knowledge of them). Thus you can conveniently use them as a guide for writing code in C++. This is the reason why I am writing a tutorial series on how to make the transition from Unreal Engine blueprints to C++.
      Learn and practice C++
      Following this tutorial, you’ll acquire new concepts of C++ programming in every chapter. Then following chapters will give you reasons to reuse and practice those same concepts. There’s no better way to wire you brain.
      Link to the tutorial: [Tutorial] Learn C++ in Unreal Engine 4 by making a powerful camera
      Please do send me as much feedback as you want. I’ll be considering every constructive remarks and taking them into consideration. Your feedback will help me to improve and update the existing chapters and to make the next one better.

      View full story
    • By BenjaminBouchet
      Learning game development in Unreal Engine could be a daunting task for someone who don’t know where to start, and a cumbersome process if you don’t organize your progression correctly. One thing commonly known by experienced developers and by people unfamiliar with coding: mastering a development language is a long and difficult task.
      From blueprints to C++ in Unreal Engine
      If you want to learn fast, you need a good learning strategy. Unreal Engine contains a very powerful tool which you can use to learn C++ faster: its blueprint system. Blueprints are extremely easy to learn (and you may already have a good knowledge of them). Thus you can conveniently use them as a guide for writing code in C++. This is the reason why I am writing a tutorial series on how to make the transition from Unreal Engine blueprints to C++.
      Learn and practice C++
      Following this tutorial, you’ll acquire new concepts of C++ programming in every chapter. Then following chapters will give you reasons to reuse and practice those same concepts. There’s no better way to wire you brain.
      Link to the tutorial: [Tutorial] Learn C++ in Unreal Engine 4 by making a powerful camera
      Please do send me as much feedback as you want. I’ll be considering every constructive remarks and taking them into consideration. Your feedback will help me to improve and update the existing chapters and to make the next one better.
  • Advertisement
  • Advertisement
Sign in to follow this  

C++ 32bit to 64bit conversion of file fixup code

Recommended Posts

I'm converting a load of c++ code from 32 bit to 64 bit, and have run up to the predictable snag of fixup (relocation) pointers in binary files.

Essentially there are a bunch of pointers in a binary file, but when saved on disk they are relative to the start of the file. Then on loading, the pointers are 'fixed up' by adding the address in memory of the start of the file to the offset, to give an absolute pointer which can be resaved in the memory location, and used at runtime as normal pointers.

This is great but has so far been relying on the offset and pointer being 32 bit. The files are unlikely to be anywhere near 4 gigs so the offsets don't *need* to be 64 bit.

My question is what would be best (or rather what do most of you guys do) for this situation? One particular quirk is that the code needs to compile and run fine as 32 bit and as 64 bit as it needs to run on both classes of device, and the binary files must be the same.

  1. The most obvious solution is to store all the offsets / pointers in the binary file as 64 bit. This would mean re-exporting all the binary files, but this is doable (even if somewhat of a pain). This would simplify things for 64 bit version, and require only slight modification for 32 bit. The downside is the file sizes / size in memory would be bigger + any cache implications.
  2. Keep the pointers as 32 bit offsets and do the pointer addition on the fly as the parts of the data need to be accessed. The files are kept the same and the only cost is the extra pointer arithmetic at runtime. I have a vague memory of seeing a presentation by a guy who did such relocation on the fly and found there was very little runtime cost.

There also appears to me the question, even with 64 bit pointers, are they ever going to be more than a 32 bit value if the program is using a lot less than 4 gigs? I'm assuming yes as the address space may be past the first 4 gigs, and all the virtual memory address space / paging / randomization that goes on, but I just thought I'd check that assumption, as I'm not well versed on the low level details.

Edited by lawnjelly
Missed out 'best' in sentence :)

Share this post


Link to post
Share on other sites
Advertisement

Imho, option 1 would be useful if you only need to target 64 bit, ie you'd do the same trick that you already did, but in the 64 bit world. In this case, option 1 breaks the 32 bit version as much as the current version is broken in 64 bit.

When you resize a pointer, all data behind it shifts. That means pointers pointing into that data must adapt to that shift too. With 2 platforms, you're going to have that problem in one of them, if you use native pointers in the data.

 

So option 2 seems much more useful to me. I am not sure how you need an offset in the file that is longer than 32bit. Surely the pointers only point within the data itself, right? I mean, pointing outside the block that you load imply the relative position of the block and the thing it points to, stays the same upon loading, which isn't really the case if you allocate data separately.

You can easily test this, load the file, and check that all offsets are between 0 and the size of the data file.

Share this post


Link to post
Share on other sites

Yes I admit, given the need to keep support for 32 bit, I'm inclined towards option 2. As you say none of the offsets require being more than 32 bit as they are relative addresses within the file.

Rather than keeping the pointers as offsets in the 32 bit version, maybe I can make the fixup routine a 'no op' in the 64 bit version, and access the pointers through an accessor function that simply returns the pointer in the 32 bit version, and does the offset + start calculation in the 64 bit version.

Share this post


Link to post
Share on other sites

I use option #1 and #2 in different situations :)

If re-exporting is a pain, fix your build pipeline :D

I have multiple template types for #2. Storing offsets from the start of the file is actually harder to deal with at runtime, because you can't just use the variable like a pointer, as "dereferencing" requires you to know the address of the beginning of the file.

What I usually do is to store offsets from the current file-cursor (offsets from the "fake pointer" field itself), which is simpler to use at runtime.

e.g. For offsets-from-cursor, I use this template which has overloaded operators to make it look just like a regular pointer at runtime.

template<class T, class Y=s32> struct Offset
{
	const T* NullablePtr() const { return offset ? Ptr() : 0; }
		  T* NullablePtr()       { return offset ? Ptr() : 0; }
	const T* Ptr() const { return (T*)(((u8*)&offset) + offset); }
		  T* Ptr()       { return (T*)(((u8*)&offset) + offset); }
	const T* operator->() const { return Ptr(); }
		  T* operator->()       { return Ptr(); }
	const T& operator *() const { return *Ptr(); }
		  T& operator *()       { return *Ptr(); }
	static uint DataSize() { return sizeof(T); }
	bool operator!() const { return !offset; }
	Offset& operator=( void* ptr ) { offset = ptr ? (Y)((u8*)ptr - (u8*)&offset) : 0; return *this; }

	Y offset;
};

Or this one for offsets-from-beginning-of-file, which isn't compatible with operator overloading:

template<class T, class Y=u32> struct Address
{
	const T* Ptr(const void* base) const { return (T*)(((u8*)base) + address); }
		  T* Ptr(const void* base)       { return (T*)(((u8*)base) + address); }
	uint DataSize() const { return sizeof(T); }

	Y address;
};

And this is probably over-complicated, but it works for option #1:


template<class T> struct Pad32to64
{
	union{
		T   data;
		u64 pad;
	};
	operator       T&()       { return data; }
	operator const T&() const { return data; }
	T operator -> () { return data; }
	const T operator -> () const { return data; }
	bool operator!() const { return !data; }
	operator bool() const { return !!data; }
	Pad32to64<T>& operator=( const T& o ) { data = o; return *this; }
};
template<int size, class T> struct Select64Wrapper {};
template<class T> struct Select64Wrapper<4,T> { typedef Pad32to64<T> Type; eiSTATIC_ASSERT(sizeof(Type)==8); };
template<class T> struct Select64Wrapper<8,T> { typedef           T  Type; eiSTATIC_ASSERT(sizeof(Type)==8); };
template<class T> struct PadTo64
{
	typedef typename Select64Wrapper<sizeof(T), T>::Type Type;
};

struct Test {
  PadTo64<int*>::Type myPointer;
};

 

Share this post


Link to post
Share on other sites

Ah some very nice prebuilt solutions there from Hodgman, many thanks. :) I might have to steal some of those ideas, using the offset from the "fake pointer" looks like it works well! :D

Share this post


Link to post
Share on other sites

What's important, from a design point of view, is making the whole system (file format, writer, reader) oblivious to pointer size. You shouldn't have the 32 bit version and the 64 bit version, but a single implementation that compiles correctly and behaves identically on all platforms.
Offsets are measured in bytes and represented as 32 bits in the file because the file is limited to 2 or 4 GB and file content is unaligned (two assumptions that in some cases could be changed to allow smaller offsets), not because pointers are 32 bits.
Pointer arithmetic in code needs to convert a 32 bit integer, (i.e. int32_t) to a ptrdiff_t and vice versa, at worst using type traits from the standard library (numeric_limits<ptrdiff_t> from <limits>) to check for overflow.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Advertisement