Seminar 18 Guitar Hero: staircase and iterator functions

18.1 Getting the difficulty just right

In game design, one of the hardest things to get right is difficulty. Make your game too easy and it will be boring. Make it too hard and only hardcore fans will play and only for an achievement. Thus, you would like to make your game hard enough to push a player to the limit but not much harder than that, so not to frustrate them. One way to solve this conundrum is to create different preset difficulty levels, something we did in our Snake game. An alternative way is to make a game that adapts its difficulty to the player.

The same is true for psychophysical experiments. You want to test ability of your participants to perform a certain task at their limit for one simple reason: At this threshold point influence of any additional factor, whether positive or negative, is most pronounced. For example, use an unusual stimulus configuration or increase attentional load and performance will drop. Allow to preallocate attention via cuing or use a prime that is congruent with a target and performance will improve. Of course, these manipulations will have the same overall effect also when the task is particularly easy or maddeningly hard but it will much more difficult to measure this effect. It is one thing if performance drops from 75% to 65% than if it goes from 98% to 95% or from 53% to 52% (here, I assume that 50% is chance level performance) or vice versa. The silliest thing you can do is to hope that performance will allow you too see the effect of the factors that you manipulated. In things like these, knowledge and careful design is definitely superior to hope.

Thus, you want performance of your participants to be approximately in the middle between the ceiling (100% performance, fastest response times, super easy) and the floor (chance level performance, slowest response times, super hard or even impossible). But how do you know where this magic point for a particular person is? Particularly, if the task is novel so you have little information to guide you. The solution is to adjust the difficulty on-the-fly based on participant’s responses. For example, If you have a two-alternatives-forced-choice task, you can use a two-up-one-down staircase (difficulty increases after two correct responses and decreases after one mistake) that targets 70.7% performance threshold. There are different methods and even different ways to use the same core method (e.g., does the step stays constant or changes, what is the run termination criteria, etc.), so it is always a good idea to refresh your memory and read about adaptive procedures when designing your next experiment.

In our game, we will use a very simple 3-up-1-down staircase: get the three responses correct on a row and things get faster, make a mistake and the game slows down. We’ll see how fast you can go!

18.2 Guitar Hero

Today, we will program Guitar Hero game. In the original game, you must play notes on guitar-shaped controller at the right time, just like when you actually play music on a guitar. On the one hand, it is a straightforward and repetitive motor task. On the other hand, take a fast and complicated music piece and it’ll take many minutes or even hours of practice to get it right. It is a lot of fun, as music cues and primes your responses. The same idea of music-synchronized-actions was used in Raymon Legends music levels where jumps and hits are timed to drums or bass. It is a bizarrely cool dance-like sequence and a very satisfying experience, also when watching pros to do it (I happened to have one in my household).

We will program this game (sans Guitar and Hero) and you can see it in the video below. The player must press a correct key (left, down, or right) whenever the target crosses the line. Pressing it to early or too late counts as a mistake. Of course, the faster the targets go, the harder it is to respond on time and with a correct key. As I wrote above, we will use the 3-up-1-down staircase procedure to control for that.

As per usual, we will take gradual approach:

• Boilerplate code
• Create a class for individual moving targets
• Create a timed-response task class that will create them (using cool generators), dispose of them, check the response, and adjust staircase.
• Add bells-and-whistles like score and time limited runs.

18.3 Boilerplate

Create our usual boilerplate code in exercise01.py (you will use the same files for classes but use versioning of the main file, it makes it easier for me to check your code):

• Import what is needed from PsychoPy
• Create a window. I’ve picked 640×480 but choose whatever looks good on your screen.
• Create our usual main game loop with gamover variable, flipping the window, and checking for escape button press.

Put your boilerplate code into exercise01.py.

18.4 Target and TimedResponseTask classes

Our main work horse will be TimedResponseTask class. It will spawn a new random Target at random intervals (which will depend on speed), pass speed information to moving targets, and remove targets, once they disappear below the screen. The Target class use visual.Rect⁵⁰ with some extra bells and whistles to make it appear at the right location, move at the right speed, change it line color (indicating a correct response), compute whether it is already off the screen, etc. We will start with a single target first.

18.5 Target class: static

First create a Target class: a colored rectangle in one of the three positions that starts at the top of the window and moves down at a specific speed. Its constructor should take PsychoPy window as a parameter (you will need it to create the rectangle attribute), position index (from 0 to 2), and speed (in "norm" units per second). The only thing we need to do right now in the constructor is to create a color rectangle (see below) and store both ipos and speed as attributes for later use. In addition, define a score attribute and set it to None. This will hold the score the participant got for this target and None means that it has not been responded upon.

The second parameter — position index — determines the horizontal position of the target and its color (to make targets more fun and distinct). For my code, I decided to make rectangle 0.4 norm units wide and 0.1 norm units high. The leftmost red rectangle (for ipos 0), is centered at -0.5, the middle green one is dead center, and the rightmost blue rectangle is centered at 0.5. I’ve defined all these as TARGET_ constants (e.g., TARGET_COLOR = ...), so think about how you can compute both color and position without using if-else statements. Also, think about the y-position of the rectangle, so it appears right at the top of the window.

The second method you need is draw() which simply draws the rectangle. Test it by creating a target at one of the position (or three targets at all three positions) and drawing them in the main loop. You should get nice looking but static rectangle(s).

Put updated code in exercise02.py and create the class Target in a separate file.

18.6 Target class: moving

Our targets fall down at speed defined by their speed attribute. Later on, we will change that attribute dynamically to speed up or slow down their fall.

For the actual falling down, implement a new method, call it fall(), that will update target’s position on every frame. The speed is in norm units per second, thus, to compute the change in vertical position you also need to know the how much time in seconds has elapsed since last position update. The simplest way to do this is by using a Clock class. You create it as an attribute in the constructor and then, in the fall method you use its current time to compute and apply a change in vertical position of the rectangle. Don’t forget to reset the clock after that!

Include fall method call in the main loop and see how the target falls. Experiment with falling speed!

Put updated code in exercise03.py and update the class Target.

18.7 Iterator/Generator functions

In the next section, we will create a TimedResponseTask class that will generate targets at a random location and after a random interval. We can, of course, do it directly in the class but where’s fun in that?! Instead, we will use this as an opportunity to learn about iterator/generator functions. You already know how to use duck typing to turn any class into a iterator by defining special method __iter()__ and __next__(). But a function can also be an iterator if it uses yield instead of return statement to, well, yield a value. It yields it, because the function itself returns an iterator object that you can iterate over in a for loop or via next() function. Importantly, yield “freezes” execution of the function and the next time you call the function it continues from that point rather than from the start of the function. Once you reach the end of the function, it automatically raises StopIteration() exception, so you don’t need to worry about how to communicate that you ran out of items. It may sound confusing but it really simple. Here an example to illustrate this:

def iterator_fun():
    yield 3
    yield 1
    yield "wow!"
  
# function returns an iterator, not a value!
print(iterator_fun())

# iterating via for loop

## <generator object iterator_fun at 0x00000000273DBA98>

for elem in iterator_fun():
    print(elem)
    
# iterating via next(), note you use an iterator object 
# that function returned, not the function itself!

## 3
## 1
## wow!

an_iterator = iterator_fun()  

# now you can use an_iterator to get a next item from it
print(next(an_iterator))

## 3

print(next(an_iterator))

## 1

print(next(an_iterator))

## wow!

# next call will raise an exception StopIteration(), so I do not run it here.
print(next(iterator_var))

This format makes writing iterators very easy, just yield whatever you want in an order you want and Python will take care of the rest. You can also yield in a loop, inside an if-else statement, etc. Look at the code below and figure out what will be printed out before running it.

def iterator_fun():
  for e in range(4):
    if e % 2 == 1:
      yield e

for item in iterator_fun():
  print(item)

For our TimedResponseTask class, we will need two generators. They are generators rather than iterators because both will be endless. One that generates a random delay until the next target and one that generates a random target position (0, 1, or 2). Implement both in a separate file (I called it generators.py).

The time_to_next_target_generator() function should take a tuple of two float values, which define shortest and longest allowed delays, as a parameter and yield a random number within this range in an endless loop. We need the endless loop (while True: will do) because we do not know how many values we will need, so we just generate as many as needed.

The next_target_generator() will be a bit more interesting. It can just return a random.choice from 0, 1, and 2 but where is fun in that? Instead, we will make it a bit more complicated to ensure that all three targets appear equal number of times within 3N trials, where N will be a parameter of the generator function. This would ensure random, reasonably unpredictable but balanced targets in the short run. Remember, in the long run random choice will always give us a balanced uniform distribution but there is not such guarantee for the shorter runs of a few trials. First, you should create a list where each target appears N times (think how you can do it using range(), list() and *). Then, create an endless loop (again, we don’t know how many values we will need) in which you 1) shuffle elements of the list, 2) yield one element at a time via for loop. Once you run out of elements, you shuffle them again and yield one by one again. Then repeat. And again, and again. Endless loop!

I would suggest creating and testing both function in a Jupyter notebook first and then putting them in a separate file (e.g., generators.py). Be careful if you decide to use a for loop instead of next() for testing. Remember, both a generates and will never run out of items to yield for a for loop!

Put both generators into generators.py.

18.8 TimedResponseTask class

Now we are ready to create the TimedResponseTask class. For our first take, it will create targets at a random location (next_target_generator()) after a random interval (time_to_next_target_generator()) plus take care of moving and drawing all of them. More bells and whistles (disposing of targets that went past the screen, changing the speed, checking response validity, etc.) will come later.

For the constructor, we definitely need PsychoPy window as a parameter, because we need it every time we create a new target. We can also pass initial speed, a tuple with range for time intervals between targets for time_to_next_target_generator(), and number of target repetitions for the next_target_generator(). We could define those as constants but passing them as parameters will make your code more reusable. Store window and speed as attributes, create attribute targets and initialize it to an empty list, create attributes for both generator objects using the appropriate parameters.

Also create a speed_factor attribute and set it to 1. We will use it later to control both the speed of motion and how frequently the targets are generated. The higher is the factor, the faster targets move and the shorter is the interval to the target and vice versa.

Finally, we need CountdownTimer that will count the time down to the moment when we need to generate a new target. Initialize it it to the next() item from time-to-next-target generator (remember, you need to use the attribute, which is a generator object that function returned, not the function itself) divided by scale_factor (the faster is the speed of the game, the shorter is the interval to the next target).

Now we need to add three methods draw, update, and add_next_target. The first one is easy, it simply draws all targets in a for loop. The second is also easy, it makes all targets fall plus, after the loop, it should call add_next_target method. The latter should check whether the timer has counted down to zero (or below, remember, time is float, so do not expect it to be exactly zero) and, if that is indeed the case, create a new random target (get the next() position from the position generator and remember to pass speed times speed_factor!), add it to the list of targets, and reset the timer using next() item from the time generator (remember to divide it by speed_factor!).

In the main file, create TimedResponsTask object (use a name you like) and call its draw and update methods in the main loop. You should see targets appearing at random and falling down consistently.

Put updated code in exercise04.py and create the class TimedResponseTask.

18.9 Disposing of unneed targets

Currently, our targets keep falling down even when they are below the screen. This will not affect the performance immediately but it will be taxing both memory and CPU, so we should dispose of them. In the Target class, create a new read-only (computed) property is_below_the_screen that returns True if the upper edge of the target is below the lower edge of the screen (False otherwise, of course and you definitely do not need if-else!).

Next, in the update method of TimedResponseTask, add a second loop (or modify the existing loop) where you delete any object that is_below_the_screen (if you are not sure how to do it, take a look back at our Space Invaders game).

For debugging, run the main code, wait until at least one target falls below the screen, put a break point and check targets attribute. Its length should match the number of visible targets, not of the total generated targets.

Update classes Target and TimedResponseTask.

18.10 Finishing line

Add a new visual attribute to the TimedResponseTask that is a horizontal line. The task of the player will be to press a corresponding key whenever a target crosses (overlaps with) the line. For now, create it as an attribute in the constructor (pick the vertical location you like) and draw it inside the draw method.

Update class TimedResponseTask.

18.11 Response

Now the real fun begins! We will allow a player to press keys and check whether a corresponding target is on the line. For this, we need new methods for both Target and TimedResponseTask classes. For the Target, implement a new method class overlaps() that will take a vertical position (of the finishing line) as the only float number parameter. In the method, first you check that the score attribute is None. If it not none that means that the player already responded on to the target and they are not allowed to respond to it twice. If it is None, compute a score using the following formula: \[score = int \left(10 - 10 \cdot \frac{|y_{target} - y_{line}|}{h_{target} / 2} \right)\] where \(y_{target}\) is the vertical center of the target, \(y_{line}\) is the vertical position of the line (you get it as a function parameter), \(h_{target}\) is height of the target, \(||\) means absolute value (use fabs function from math library for that), and 10 is an arbitrary scaling factor (you can use any integer). Study the formula and you will see that score is 10 if the target’s center is right on the line but decreases linearly with any displacement. Once the target is off the line, the score becomes negative. We convert it to int, because we want simple scores (floats look messy for this). Compute the score and store in a temporary local variable. If the value is positive, that means success, so you should store this value now permanently in the score attribute, change line color of the rectangle to white (to show the player that they got it right), and return True (yes, target does overlap with the line!). For all other outcomes, you return False. This means that either the response was already made or the target does not overlap with the line.

In the TimerResponseTask class, we need a new method check() that will take position of the target based on the key press (so if a player pressed left key, the position will 0, down is 1, and right is 2). Loop over targets and if a target’s position (ipos attribute) matches the position of the key press (parameter of the function) and target overlaps with the line (the overlaps() method returns Treu), return the score attribute of that target. Note that the condition order is important here! You need to check for the overlap only if target position matches the key. If you ran out of targets to check that means that the player pressed a wrong key or at the wrong time, so you should return 0 (means “mistake”).

In the main loop, add "left", "down", and "right" to the key list of getKeys() call. Then, if any of these three keys are pressed, translate that into a position, respectively, 0, 1, and 2 (think how you can do it without if-else via a dictionary), and call the new check method of the TimedResponseClass. Test the code, targets’ edges should turn white, if you time your key press correctly!

Put updated code in exercise05.py, update Target and TimedResponseTask classes.

18.12 Score

Playing is more fun when you can see how well you are doing. Let us add a simple score indicator that is updated with response score. You already know how you can do it via TextStim stimulus but you also already know how you can inherit from a base class and extend its functionality. This is what we will do here, as the class will record and draw the score (that part is covered by the inheritance).

Create a new class (I have called it ScoreText) that inherits from TextStim. In the constructor, you need to create an integer attribute that will hold current score and initialize it 0. Plus, call ancestor’s constructor via super().__init__(...) to initialize and place the text stimulus (I’ve picked top left corner). Think about parameters that the constructor and ancestor’s constructor need.

Next, we need to update the score (both its numeric form and the text that we draw) every time participant presses a key. We could implement the code outside of the class but that is a fairly bad idea, as it puts class-related code elsewhere. We could also implement a “normal” method, e.g., add() that will take care of that. Instead, we will implement a special method iadd that allows to “add to” the object. It takes a single parameter (in addition to the compulsory self, of course), performs “adding to the self” operation (whatever that means with respect to your object, can be mathematical addition for an attribute, concatenation of the string, adding to the list, etc.), and returns back the reference to itself, i.e., returns self not a value of any attribute! Here’s how it works:

class AddIt():
    def __init__(self):
        self.number = 0
        
    def __iadd__(self, addendum):
        self.number += addendum
        return self # important!!!


adder = AddIt()
print(adder.number)

## 0

adder += 10
print(adder.number)

## 10

Implement that special method for your class, so we can do score_stim += timed_task.check(...). Remember, you have to update both numeric and visual representations of the score in that method! Add the score to the main code.

Put updated code in exercise06.py, create ScoreText class.

18.13 Staircase

We will implement the staircase as part of the TimerResponseTask class, so it can speed up and slow down itself. For this, we will need an attribute that counts number of consecutive correct responses (I, typically, call it correct_in_a_row or something like that). Create and initialize it to zero in the constructor.

Next, create a new method staircase() that will take a single parameter (beyond self) on whether the response was correct or not. If it was, increment correct_in_a_row by one and check whether it reached 3. If it did, increase the speed_factor by multiplying it by some chosen factor (I’ve picked 1.3) and resetting correct_in_a_row to 0. Alternatively, if the response was not correct, divide speed_factor by the same number (e.g., 1.3, so slowing things down) and again, reset correct_in_a_row to 0. After that, loop over all targets and update their speed based on speed and speed_factor attributes.

You need to call this method inside the check method, think then and how.

Update TimedTaskResponse class.

18.14 Limiting time

Let us add a competitive edge by limiting the run time to 20 seconds (you can pick your own duration, of course). Create an additional outer loop, so that the game can be played many times over. Once the round is over, show the latest state (redrawing all game objects) plus the “Round over” sign and wait for the player to press either escape (then you exit the game) or space (to start the next round). Remember to recreate all game objects anew for the next round (or create a reset method for all of them).

Put updated code in exercise07.py, create ScoreText class.

18.15 This is just a start!

As per usual, think about how you can extend the game. A clock showing the remaining time is definitely missing. Auditory feedback would be nice. More positions? Random colors to confuse a player?

---

50. Inheriting from visual.Rect would’ve make it even simpler but, due to so technical glitch, unfortunately that does not work.↩︎