The pros, cons, how's and when's of a closed alpha
My game is now reaching a point where there is enough rudimentary content to start a closed alpha test.
I am thinking of setting this up as a private url to the games download link, distributed as a zip containing all 750 mb of debug game.
I expect to have no more than 20 testers and use something like jira with service desk to log and track bugs/features as I have experience with this software and it can be purchased really cheaply as I am a one man band.
I do have a few concerns about this approach.
I am worried that the game url or files may be leaked, giving people who obtain a copy the false opinion that it's a half arsed finished product, and potentially creating an army of testers I couldn't ever hope to manage. Or, that maybe this is too early to be starting releases at all. after all the game is barely playable. However going on past experience in open source, people love to get their hands on work in progress and even if it was just a sprite bouncing around a 2d map (which it isn't!) generally people have always offered constructive feedback.
So, what are your thoughts on this. Would you do anything differently, or perhaps not at all? Would you offer alpha releases and how would you select the testers? At this point I mainly want dedicated games players not professional testers (I know a couple of these who have offered their time for free but I think the game needs more substance to it before I unleash them on it)
Thoughts more than welcome!
Hey, congrats on getting to the alpha stage, many ideas don't make it to this point!
What I did was playtest my game myself, to the point, where everything seemed to work fine for me - that is when you need testers who can confirm that everything is indeed good (which is never true).
Then, I did what you are trying to do now - upload the installer to a random hosting website and throw links everywhere I could, which was a handful of friends with the minimum required hardware. When the hard deadline for the release was approaching and I still needed more tests, I ended up putting the download link on facebook. Suffice to say, I got the testing data I needed in time. In the end, who cares who gets a half-finished product from you? You don't have to manage all the testers, just those you want to. I hope that helps ;)
Well, you could have a password system in place for beta testing. The devs of Interloper (On Steam now) did that.
Require the game to connect to a server to play (and have a chunk of the gameplay logic on the server). It's the only way to be sure.
Thanks everyone!
There are some good ideas here so far, I may just leave the download unprotected, and if someone wants to share the game with someone else that's fine, as Ryder052 said I can pick and choose who i listen to bug reports from.
For issue tracking, do you think Jira and Service Desk will suffice? I've heard of other solutions such as Trello which either require monthly subscription, or I am not familiar with to the same level. Some of these seem to be more generic project management apps rather than a developer issue tracker.
Swiftcoder, there isn't really any part of the game which could justify needing online content. It is a single player roleplaying game. Unless i just had an online check for purposes of verification, having the file expire every month forcing players to fetch a new version, this would make it less likely the alpha releases aren't distributed widely as a full game.
I could also make it very clear with a splash screen that it is alpha or beta content and might not reflect the final version.
Thoughts?
I could also make it very clear with a splash screen that it is alpha or beta content and might not reflect the final version.
Thoughts?
That sounds like a very good idea or maybe a visible watermark throughout maybe a little 'alpha version:xxxxxx' in one corner, It's relatively simple to implement.
So you should not be scared if anything "leaks" because there is no content, just a mechanisem to test.
- Testing is good. I prefer giving the game to people I know so I get feedback faster.
- If your approach is to use online testing, you should block the game with a beta key, so it will only be for testing purposes.
I've recently had a "private beta" release of Curver, a tool for creating line-art with an intuitive work-flow. I've learned a few things, and while Curver is not a game, I think it's reasonable to assume the experience should not be too different.
Before the release, I knew of a lot of things that needed to be implemented and fixed before the application is ready for use, but I felt that I needed to hear from people. I've read a couple of articles about the benefits of a private beta release for small companies (and lone developers such as myself). And now after the release I think I've benefited from it a lot, but not in exactly the same way I initially anticipated.
First, I created a small php application and called it "beta-access". It's a 1-page application where I generate and view beta access keys. When Curver starts, it asks the user for a beta access key, contacts the beta access application on the server, which will respond with a simple "yes" or "no". Nothing fancy here - just a simple measure to prevent my application from going fully public before I intend to. If you'd like, I can provide you with this "beta access" application.
Then I setup forums on my site, and added a feedback widget to collect ideas and bugs on a special "feedback" page.
Then I posted a link to my application and a short video about it on polycount.com - it's a mature community of great artists (much like gamedev.net but for art), and although its focus is on 3D art, I thought there has to be a few interested people who would be willing to test it and give valuable feedback. The initial response was amazing, and I felt that the development of Curver has gained great momentum with a great community of testers. Here is the post on polycount.
I thought 20 or 30 people should be enough. In a matter of a couple of days, about 30 people sent emails requesting beta access, which I provided to them. Then I changed the post to inform people that the beta is now closed.
The first piece of feedback I received was by email - it was detailed and thorough, it was AWESOME! The guy went through the trouble of recording the screen, cropping the video, and uploading multiple gif files to demonstrate what he was talking about. Then another user sent more feedback, and that was it!!! Note that your testers will probably choose the feedback method that they chose, not the one you choose for them. I asked the guy to copy his feedback from the email to the feedback widget, which he did, but yeah, expect to receive feedback wherever the testers choose to post them. One tester even posted her not-so-constructive-feedback on the polycount thread.
Now it's been about 20 days. Of the 31 people who received beta access keys, 2 have provided great feedback, 13 have logged in and tried the application but provided zero feedback, and the rest did not even bother running the application it seems!
Maybe I did not make it clear that the purpose of the beta release was specifically to collect feedback, or maybe people expected more functionality and were disappointed with an application that is still not mature enough for real world usage and quietly went away. Maybe people expected that they will receive a fully working product and might need to just occasionally report a bug here or there instead of becoming involved in the decision making of the application, in designing it, and in helping with the drawing of its road-map. Maybe I should have called it "private alpha" instead of "private beta". But to me it was beneficial nonetheless - the workflow (project plan) of my application is now clear and well organized, I now sit at the computer and know what really needs to be worked on and what needs to be pushed back. I now know the general idea of the application has potential for success, I've fixed a few bugs, enhanced a few things, and I find myself accomplishing more with my time after the release than before, where I often found myself staring at the monitor wondering what the hell it is that I'm doing.
This has been my experience. Your mileage may vary. Good luck man.
13 have logged in and tried the application but provided zero feedback
It sounds like I might need metrics. I should have some simple system that sends system details, play time and average fps to my server for analysis, maybe some flags to indicate what general areas they explored.
That way those who don't email and provide personal feedback can still be useful to the effort...
Thanks for the suggestion!