I recently bought my first Bamboo, a Wacom tablet that recognises letters from shapes drawn with the stylus. It tickled memories of my first experience with gesture-controlled application: using mouse gestures, web browsers such as Maxthon (and later Opera) allowed users to quickly move back and forth through webpages in the history, switch between different tabs, and so on. I was facinated by its neat user interface, as it takes away traditional mouse clicks. Of course, sophisticated gesture-controlled devices such as the Kinect, iPad and iPhone are now available – but it all started with the good old PC. In this tutorial, you’ll learn how to develop a photo gallery that recognises singular mouse gestures.

Final Result Preview

Let's take a look at the final result we will be working towards. To pan the photo galley in the four main directions, click and drag the mouse in the relevant direction. To scale a photo, drag the mouse South-East to scale up and drag the mouse North-West to scale back to the default size.

(Note: this doesn’t snap to center the photo when panning; it’s sensitive to the length of the line that you draw.)

Step 1: Tutorial Flow

Here’s what you’ll learn in this tutorial, and the order in which you’ll learn it:

• Vector interpretation of mouse gestures
• Hard coded implementation of mouse gestures
• Class to detect singular mouse gestures
• Sample application (photo gallery) using mentioned class

Step 2: Gesture Detection: Vector Analysis

It is important to understand the vector math involved in detecting mouse gestures. Having understood detection of a single direction, one can easily extend the understanding to apply to all eight directions.

The Flash presentation below shows steps of detecting a single mouse gesture to due right. To scroll through frames in the Flash presentation below, (mouse down – mouse move – mouse up) in any of the following directions:

• Eastward to scroll frame forward
• Westward to scroll frame backward
• Northward to jump to last frame
• Southward to jump to first frame

Step 3: Gesture Detection: Angle Alleviations

Implementing Step 2 will be easy. However, chances are 90% that users' gestures will fail. Diagram below shows commonly commited gestures (middle); they seldom comply with a rigid Vector pointing to due right (left). Thus, it's better to give alleviation for gesture inaccuracies (right).

For example, we can give an alleviation of 30° on both sides of the Vector pointing due right so that angle of any gesture's Vector that falls within that range will be accepted and interpreted as a gesture to due right.

Step 4: Gesture Detection: Sample Implementation

Below is an implementation of gesture detection to due right. Press down on mouse, move mouse to the right, and release mouse within the Flash presentation below. Try to gesture a little off the absolute right to check out the implementation of alleviation.

Step 5: Variables

Lets look at the variables on our hard-coded implementation in Step 4. I’ve highlighted the important Vector2D variables. Take note of comments I’ve placed at the end of each variable.

private var t:TextField;
private var earlier:Vector2D		//store mouse location upon first click
private var latter:Vector2D		//store mouse location upon release
private var RIGHT:Vector2D = new Vector2D(1, 0);	//vector of absolute RIGHT

Step 6: Hard-Coded Implementation

I assume you already know the basics of putting a TextField into your project so I shall focus on the ActionScript implementation of mouse gesture. The implementation as indicated below is heavily commented. Important Vector calculations are highlighted as well. I encourage readers to examine these comments, especially those highlighted, to understand operations at different events at runtime.

(The Vector2D class is the same one I’ve used in previous tutorials, like this one.)

public function HardCoded()
{
	//Creating a textbox
	t = new TextField();
	t.selectable = false;
	t.width = 300;
	t.x = stage.stageWidth/2;
	t.y = stage.stageHeight - 30;
	addChild(t);

	//Start of gesture detection
	stage.addEventListener(MouseEvent.MOUSE_DOWN, start);
}

//Register mouse location upon mouse down
private function start(e:MouseEvent):void
{
	//Register mouse suppress location
	earlier = new Vector2D(e.localX, e.localY);

	//Start to draw line
	graphics.lineStyle(3);
	graphics.moveTo(earlier.x, earlier.y);

	//add mouse move and mouse release listeners
	stage.addEventListener(MouseEvent.MOUSE_MOVE, move);
	stage.addEventListener(MouseEvent.MOUSE_UP, up);
}

//Draw gesture upon mouse move
private function move(e:MouseEvent):void
{
	graphics.lineTo(e.localX, e.localY);
}

//Evaluate gesture upon mouse release
private function up(e:MouseEvent):void
{
	//Register mouse release location
	latter = new Vector2D(e.localX, e.localY);

	//Calculating vector of mouse gesture
	var result:Vector2D = latter.minus(earlier);

	//Calculating angle from absolute RIGHT to gesture vector.
	var deviation:Number = RIGHT.angleBetween(result);
	deviation = Math2.degreeOf(deviation);

	//Interpreting gesture with alleviation
	if (Math.abs(deviation) < 30) 		t.text = "RIGHT gesture detected";
	else					t.text = "";

	//Clear screen of previous drawing.
	graphics.clear();

	//remove mouse move and mouse up listeners
	stage.removeEventListener(MouseEvent.MOUSE_MOVE, move);
	stage.removeEventListener(MouseEvent.MOUSE_UP, up);
}

Step 7: Summary

For clarification purposes, here’s a summmary of the hard-coded implementation:

1. Upon mouse down, begin gesture detection.
2. On mouse move, update the latest location of mouse pointer.
3. On mouse up, evaluate the whole gesture since (1).

Step 8: Gesture Angle Alleviations

Making an accurate gesture using a mouse is hard. It’s hard to make straight lines (East, South, West, North) but it’s even harder to make diagonal lines (South-East, South-West, North-West, North-East) because we have to estimate that extra 45°. So I have given diagonal lines more alleviations than straight lines. Notice the larger grayed angle for diagonal Vector compared to that of the straight Vector.

Step 9: Gesture Sensitivity

 I'd to point out another issue – sensitive gestures. Sensitive gestures identifies gesture directions when mouse pointer makes the slightest shifts in location, even to adjacent pixels. Diagram below illustrate scenarios of sensitive gestures.

If the user changes his mind after mouse press and mouse releases immediately, a gesture will still be detected if his pointer makes a slightest move to the adjacent pixels. We should allow users to undo gesture detection. In this tutorial, I’ve enforced a minimum magnitude the Vector of current gesture must exceed to be valid. I’ve included a diagram as below.

Step 10: Class Variables

In order to detect singular mouse gestures I have implemented MGesture. Do download and examine this ActionScript file. I shall go through its class variables first, then the class methods.

Below is the code implementation of class variables. I’ve allowed a deviation of 10° from main directions. For example, -10°-10° is considered due East, -80°-100° is considered due South, etc. I’ve also allowed a deviation of 30° from diagonal directions. So a Vector with orientation between 15°-75° will be considered due South-East, and so on. Also, the minimum magnitude to exceed is 10px.

private var mainBox:DisplayObjectContainer;
private var directions:Vector.<Vector2D> = new <Vector2D>[
	new Vector2D(1, 0), 	//East
	new Vector2D(0, 1),	//South
	new Vector2D(-1, 0),	//West
	new Vector2D(0, -1),	//North
	new Vector2D(1, 1),	//South - east
	new Vector2D(-1, 1),	//South - west
	new Vector2D( -1, -1),	//North - west
	new Vector2D(1, -1)	//North - east
];
private var diagonals:Boolean = false;

private var _deviationFromMains:Number = Math2.radianOf(10);
private var _deviationFromDiagonals:Number = Math2.radianOf(30);
private var _minDist:Number = 10;
private var _earlier:Vector2D;
private var _latter:Vector2D;

Step 11: Numbering Directions

You may have guessed it already from reference to code implementation of directions. For clarification, here are the main directions' integer representations.

Step 12: Class Methods & Property

Below are class methods for MGesture.

Step 13: Methods

The essential methods tabled in Step 12 are all documented here. Do read through the comments.

/**
 * Initiate variables
 * @param	container where mouse is detected from
 */
public function MGesture(container:DisplayObjectContainer) {
	//setting container from which mouse is moving
	mainBox = container;
}

/**
 * Method to register initial mouse location
 */
public function start ():void {
	var startMX:Number = mainBox.mouseX;
	var startMY:Number = mainBox.mouseY;
	_earlier = new Vector2D(startMX, startMY);	//pointer location, initially
	_latter = new Vector2D(startMX, startMY);	//pointer location, to be updated later
}

/**
 * Method to update mouse location
 * @return a Vector2D of current mouse location relative to that when start() is called;
 */
public function update ():Vector2D {
	_latter = new Vector2D(mainBox.mouseX, mainBox.mouseY);
	var vecUpdate:Vector2D = _latter.minus(_earlier);
	return vecUpdate;
}

/**
 * Method to validate a gesture.
 * @param	newLoc	Vector2D to new mouse location
 * @return	null if invalid gesture, a Vector2D if valid
 */
private function validMagnitude ():Vector2D {
	var gestureVector:Vector2D = update();
	var newMag:Number = gestureVector.getMagnitude();

	//if magnitude condition is not fulfilled, reset gestureVector to null
	if (newMag < _minDist)		gestureVector = null;
	return gestureVector;
}

/**
 * Method to evaluate gesture direction
 * @return Integer indicative of direction. Invalid gesture, -1.
 */
public function evalDirections():int {
	//Pessimistic search (initialise with unsuccessful search)
	var detectedDirection:int = -1;

	//validate magnitude condition
	var newDirection:Vector2D = validMagnitude();

	//if gesture exceed minimum magnitude
	if (newDirection != null) {

		//evaluation against all directions
		for (var i:int = 0; i < directions.length; i++)
		{
			var angle:Number = directions[i].angleBetween(newDirection);
			angle = Math.abs(angle);

			//check against main directions
			if ( i < 4 && angle < _deviationFromMains) {
				detectedDirection = i;
				break;
			}

			//check against diagonal directions
			else if (i > 3 && angle < _deviationFromDiagonals) {
				detectedDirection = i;
				break;
			}
		}

		//update mouse location for next evaluation
		_earlier = _latter;
	}

	//return detected direction
	return detectedDirection
}

Step 14: Photo Gallery

Now that the MGesture class is set, we shall proceed with an demo application (photo gallery) of it. I have included a source file here. Do download and follow along. First of all, get all images into the "lib" folder in your existing project. Images I’ve used here are courtesy of my wife and daughter.

Step 15: Embed Images

Create a new Actionscript class and name it PhotoView. We shall use these images to construct our gallery.

1. Generate embed code of images. They will be recognised as a generic Class object.
2. Cast Class into Bitmap objects so that we can manipulate it further.
3. Put all these Bitmap objects in a Vector array for easy selection later.

[Embed(source = '../lib/Week3.jpg')]
private var Week3:Class

[Embed(source = '../lib/Family.jpg')]
private var Family:Class

[Embed(source = '../lib/FatherDaughter.jpg')]
private var Daughter:Class

[Embed(source = '../lib/Jovial.jpg')]
private var Jovial:Class

[Embed(source = '../lib/NewBorn.jpg')]
private var NewBorn:Class;

[Embed(source = '../lib/Posing.jpg')]
private var Posing:Class

[Embed(source = '../lib/Smile.jpg')]
private var Smile:Class

[Embed(source = '../lib/Surrender.jpg')]
private var Surrender:Class

private var list:Vector.<Bitmap> = new <Bitmap> [
	new Week3 as Bitmap,
	new Family as Bitmap,
	new Daughter as Bitmap,
	new Jovial as Bitmap,
	new NewBorn as Bitmap,
	new Posing as Bitmap,
	new Smile as Bitmap,
	new Week3 as Bitmap,
	new Surrender as Bitmap
]

Step 16: Display List Management

It is important to clarify here the management of PhotoView's display list. I’ve included a Flash presentation here. To use it, make a gesture to right or left. For further details, refer to Step 2.

Step 17: Positioning Image

In the PhotoView's constructor, we initiate all necessary display objects and position them into place. Plus, we initiate MGesture and attach events listeners to start gesture detection. I’ve highlighted the event listeners. Their details are explained over the next two steps.

public function PhotoView()
{
	panel = new Sprite();			//Initate panel
	panel.x = stage.stageWidth / 2;		//Centering panel horizontally
	panel.y = stage.stageHeight / 2;	//Centering panel vertically
	addChild(panel);

	var currentBmp:int = 0;		//Current image selected for positioning
	var bmpGaps:Number = 60;	//Spacing between images
	var bmpOnX:int = 3;		//Number of images on horizontal axis
	var bmpOnY:int = 3;		//Number of images on vertical axis

	var bmp:Bitmap;			//Bitmap object to hold image
	var container:Sprite;	//Sprite container to hold bmp

	//scrolling through Y
	for (var j:int = -1*Math.floor(bmpOnY/2); j < Math.ceil(bmpOnY/2); j++)
	{
		//scrolling through X
		for (var i:int = -1*Math.floor(bmpOnX/2); i < Math.ceil(bmpOnX/2); i++)
		{
			bmp = list[currentBmp];
			bmp.x = -1 * bmp.width / 2;	//Bitmap centered horizontally in container
			bmp.y = -1 * bmp.height / 2;	//Bitmap centered vertically in container
			container = new Sprite();
			container.x = (bmp.width + bmpGaps )* i;	//Positioning container on x accordingly
			container.y = (bmp.height + bmpGaps )* j;	//Positioning container on y accordingly
			container.addChild(bmp);			//Add bitmap into container
			container.addEventListener(MouseEvent.MOUSE_DOWN, select);
			panel.addChild(container);		//Add container into panel
			currentBmp++				//Scroll to next bitmap
		}
	}

	gesture = new MGesture(stage);	//Initiate MGesture for gesture detection
	stage.addEventListener(MouseEvent.MOUSE_DOWN, start);
}

Step 18: Selecting Image to Scale

Line 99 highlighted is not related to detection of gesture, but merely to select image for scaling & placing it on top of all other images.

private function select(e:MouseEvent):void
{
	//Setting current image to scale &
	//Placing it on top of all other images
	ImgSelected = e.currentTarget as Sprite;
	panel.swapChildrenAt(panel.numChildren - 1, panel.getChildIndex(ImgSelected));
}

Step 19: Start, End and Evaluate Mouse Gesture

First function below is executed upon mouse down. The second is executed upon mouse up. I’ve highlighted start() and evalGesture() as well as evant listeners.

private function start(e:MouseEvent):void
{
	//Start gesture detection &
	//Listen for mouse up event
	gesture.start();
	stage.addEventListener(MouseEvent.MOUSE_UP, end);
}

private function end(e:MouseEvent):void
{
	//Prepare current gesture's magnitude for animation purpose
	//implement a maximum cap on gesture's magnitude
	gestureMag = gesture.update().getMagnitude() / 2;
	gestureMag = Math.min(gestureMag, maxMag);

	//Evaluate current gesture
	direction = gesture.evalGesture();

	//Once a valid gesture is detected, perform animation
	//No further gestures will be detected until animation ends
	if (direction > -1)	{
		stage.addEventListener(Event.ENTER_FRAME, move);
		stage.removeEventListener(MouseEvent.MOUSE_DOWN, start);
	}
}

Step 20: Animating the Panel and Images

Once the directions has been detected, animation will begin. Depending on the gesture made, the whole panel may move in four directions or one single image may enlarge or shrink.

private function move(e:Event):void
{
	var currentMag:Number 

	//Motion of panel translation
	if (direction < 4) {

		//Function of easing motion
		currentMag = gestureMag * Math.cos(currentAngle += 0.1);
		if (direction == 0) 	panel.x += currentMag;
		else if (direction == 1) panel.y += currentMag;
		else if (direction == 2) panel.x -= currentMag;
		else if (direction == 3) panel.y -= currentMag;
	}

	//Motion of image scaling
	else {

		//Setting a maximum cap on motion
		gestureMag = Math.min(0.30, gestureMag);

		//Function of easing motion
		currentMag = gestureMag * Math.cos(currentAngle += 0.1);

		//Conditions to scale up:
		//Gesture is to South-East &amp;
		//Image is not scaled up already
		if (direction == 4 && ImgSelected.scaleX < 1.30){
			ImgSelected.scaleX = ImgSelected.scaleY = -1 * currentMag + 1.30
		}

		//Conditions to scale down:
		//Gesture is to North-West &amp;
		//Image is scaled up
		else if (direction == 6 && ImgSelected.scaleX > 1){
			ImgSelected.scaleX = ImgSelected.scaleY = currentMag + 1;
		}
	}

	//If angle on easing function exceeds 90 degrees/ 0.5 Pi radian,
	//stop animation &amp; enable gesture detection
	if (currentAngle > Math.PI/2) {
		stage.removeEventListener(Event.ENTER_FRAME, move);
		stage.addEventListener(MouseEvent.MOUSE_DOWN, start);
		direction = -1;		//Reset direction
		currentAngle = 0;	//Reset angle
	}
}

Step 21: Publish PhotoView

Now all is set. You may finally publish your work by pressing Ctrl + Enter on FlashDevelop. Again. here's a piece of the final product.

Conclusion

This is not the end of it. In the next part, we shall look at detection of a gesture sequence, which will be even more interesting than this part (which really just showed the basics). Do drop comments and let me know if MGesture had been useful to you, as well as any bugs, if you encountered any. Finally, terima kasih for the time reading. I’m hoping to entertain my fellow readers in Part 2.