Commit e3fd58c8 authored by Karel Rymeš's avatar Karel Rymeš

Start of the Project

Clean version of the software provided
parents
File added
File added
Here is a list of the major changes, for details, please see ABDoc.pdf or ServerClientProtocol.pdf under the ./doc folder.
================= Server client protocol changes since the IJCAI version ================
The signal byte of the termination is changed from “2” to “-1” (see section Termination in /doc/ServerClientProtocol.pdf)
A few commands are added to the server software (see section Run the Server in /doc/ServerClientProtocol.pdf)
================ Software Changes Since abV1.31 ====================================
1. The RealShape segmentation can now distinguish between hollow and solid objects (see ABDoc.pdf section Object Representation)
2. Better triangle recognition: Triangles are treated as Polygon.java while the previous version often mis-detects the triangles as circle.
3. Add findHills() in Vision.java (in the previous version, the hill is returned by the method findBlocksRealShape())5.
4. Add findTNTs() in Vision.java (in the previous version, you need to call VisionMBR.findTNTs() to get all the TNTs)
5. In Rect.java, the attributes PreciseWidth and PreciseHeight are changed to plength and pwidth, respectively. The corresponding methods are also changed accordingly.
6. The method estimateLaunchPoint (see /src/ab/planner/TrajectoryPlanner.java) now returns empty list of release points when no plausible trajectories found for that target, and in this case, the sample agent will make a 45 degree shot.
7. The isReachable method (see /src/ab/utils/ABUtil) now returns true (reachable) when a trajectory can reach the target without obstruction. Previously, the method only checks the obstruction by assuming the input trajectory always passes the target.
8. Bug Fixes:
8.1 Fix the bug that the method findBirdsRealShape() always returns unknown birds.
8.2 Fix the bug in build.xml where the main-class was set to "ClientNaiveAgent.java" (should be "MainEntry.java")
8.3 Fix the bug that the method findBlocksRealShape() returns Pigs. (Now the method returns Wood, Stone, and Ice)
================ Software Changes Since abV1.3 =====================================
1. New Vision Module (see Package ./src/ab/vision):
The new vision module is composed of two image segmentation components. One component segments an image and outputs a list of the minimum bounding rectangles (MBR) of essential objects in the image. The essential objects includes {“Sling”, "Red Bird”, “Yellow Bird”, “Blue Bird“, “Black Bird”, “White Bird”, "Pig", "Ice", “Wood", "Stone", "TNT”, “TrajPoints”}. The other component outputs real shapes instead. You can use either of them to process a screenshot.
2. Useful methods:
2.1 Get the type of the bird on the sling (see File ./src/ab/demo/other/ActionRobot.java).
2.2 Get all the supports of an object (see File ./src/ab/utils/ABUtil.java).
2.3 Test whether a point is reachable by a trajectory (see File ./src/ab/utils/ABUtil.java).
4. Code re-factoring and bug fixes.
Copyright © 2014, XiaoYu (Gary) Ge, Stephen Gould, Jochen Renz, Sahan Abeyasinghe, Jim Keys, Andrew Wang, Peng Zhang. All rights reserved.
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/
or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.
\ No newline at end of file
ANGRYBIRDS AI AGENT FRAMEWORK
Copyright © 2014, XiaoYu (Gary) Ge, Stephen Gould, Jochen Renz, Sahan Abeyasinghe, Jim Keys, Andrew Wang, Peng Zhang. All rights reserved.
This software contains a framework for developing AI agents capable of
playing Angry Birds. The framework is composed of a javascript plugin
for the Chrome web browser and Java client/server code for interfacing
to the game and implementing the AI strategy. A sample agent is provided.
* The details of the server/client protocols can be found in ./doc/ServerClientProtocols.pdf
* We provide a sample agent communicating with the server by the pre-defined protocols. The implementation can be found in ./src/ab/demo/ClientNaiveAgent.java
* We provide a wrapper class in java that can encode/decode the communicating messages. The wrapper can be found in ./src/ab/demo/other/{ClientActionRobot.java , ClientActionRobotJava.java}
* We provide a sample C++ client that demonstrates how to connect to the server and send a doScreenShot message. It can be found in ./src/ab/demo/abClientExample.cpp
=================== Commands of the server/client version =====================================================
* Please start the server first, and then start your client.
To start the server: java -jar ABServer.jar
To start the naive agent (server/client version): java -jar ABSoftware.jar -nasc [IP]
*You do not need to specify the IP when the server is running on localhost
===================== Commands of the standalone version =======================================================
java -jar ABSoftware.jar -na // run the agent from level 1
java -jar ABSoftware.jar -na [0-21] // run the agent from the specified level.
java -jar ABSoftware.jar -na [0-21] -showMBR // run the agent with the real-time vision output (MBR segmentation)
java -jar ABSoftware.jar -na [0-21] -showReal // run the agent with the real-time vision output (Real shape segmentation)
java -jar ABSoftware.jar -showMBR // show the real-time MBR segmentation
java -jar ABSoftware.jar -showReal // show the real-time real shape segmentation
java -jar ABSoftware.jar -showTraj // show the real-time trajectory prediction
java -jar ABSoftware.jar -recordImg [directory] // save the current game image to the specified directory
================================ Outline of the source files ====================================================
The src folder contains all the java source codes of the software.
The following are the files you may want to modify:
========= Files under ./src/ab/demo/ =====================
ClientNaiveAgent.java : A server/client version of the Naive agent that interacts with the server by the pre-defined protocols
NaiveAgent.java : A standardalone implementation of the Naive agent. You can run this agent without the server.
======== Files under ./src/ab/demo/other/ ================
ActionRobot.java : A java util class for the standalone version. It provides common functions an agent would use. E.g. get the screenshot
ClientActionRobot.java : A server/client version of the java util class that encodes client messages and decodes the corresponding server messages complying with the protocols. Its subclass is ClientActionRobotJava.java which decodes the received server messages into java objects.
LoadingLevelSchema.java / RestartLevelSchema.java / ShootingSchema.java : Those files are only for the standalone version. A standalone agent can use the schemas respectively to load levels, restart levels, and launch birds.
======== Files under ./src/ab/planner/ ====================
TrajectoryPlanner.java : Implementation of the trajectory module
======= Files under ./src/ab/vision ======================
Vision.java : the entry of the vision module.
VisionMBR.java : image segmenting component that outputs a list of MBRs of a screenshot
VisionRealShape.java: image segmenting component that outputs a list of real shapes of a screenshot
======= Other Useful methods in /src/ab/utils/ABUtil.java =========================================
isSupport(ABObject object, ABObject support): returns true if support directly supports object
getSupports(ABObject object, List blocksList): returns a list containing the subset of blocksList that support object
isReachable(Vision vision, Point target, Shot shot): returns true if the target is reachable (without obstruction) by the shot
====== Files under ./src/external ===========================
ClientMessageEncoder.java : encode the client messages according to the protocols
ClientMessageTable.java : a table that maintains all the client messages and its corresponding MIDs.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project default="create_run_jar" name="Create Runnable Jar for Project AB1.0 with Jar-in-Jar Loader">
<!--ANT 1.7 is required -->
<property name="lib.dir" value="external"/>
<path id="classpath">
<fileset dir="${lib.dir}" includes="**/*.jar"/>
</path>
<target name="clean">
<delete dir="src" includes = "**/*.class"/>
</target>
<target name="compile">
<mkdir dir="src"/>
<javac srcdir="src" destdir="src" classpathref="classpath"/>
</target>
<target name="jar">
<jar destfile="ABSoftware.jar">
<manifest>
<attribute name="Main-Class" value="org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader"/>
<attribute name="Rsrc-Main-Class" value="ab.demo.MainEntry"/>
<attribute name="Class-Path" value="."/>
<attribute name="Rsrc-Class-Path" value="./ Jama-1.0.2.jar json-simple-1.1.1.jar WebSocket.jar commons-codec-1.7.jar"/>
</manifest>
<zipfileset src="external/jar-in-jar-loader.zip"/>
<fileset dir="src/"/>
<zipfileset dir="external" includes="Jama-1.0.2.jar"/>
<zipfileset dir="external" includes="json-simple-1.1.1.jar"/>
<zipfileset dir="external" includes="WebSocket.jar"/>
<zipfileset dir="external" includes="commons-codec-1.7.jar"/>
</jar>
</target>
</project>
File added
This diff is collapsed.
{
"name": "Angry Birds Interface",
"version": "1",
"manifest_version": 2,
"content_scripts": [
{
"matches": ["http://chrome.angrybirds.com/"],
"css": ["style.css"],
"js": ["jquery-1.7.2.min.js", "script.js"]
}
]
}
\ No newline at end of file
/*****************************************************************************
** ANGRYBIRDS AI AGENT FRAMEWORK
** Copyright (c) 2014, Sahan Abeyasinghe, XiaoYu (Gary) Ge, Stephen Gould,
** Jim Keys,Jochen Renz
** All rights reserved.
**
** This software is distributed under terms of the BSD license. See the
** LICENSE file in the root directory for details.
*****************************************************************************/
(function() {
var sock = null;
function connect() {
if (sock !== null) {
return;
}
console.log('Connecting...');
sock = new WebSocket('ws://localhost:9000/');
sock.onopen = function() {
console.log('Connected!');
};
sock.onclose = function(e) {
sock = null;
console.log('Connection closed (' + e.code + ')');
reconnect();
};
sock.onmessage = function(e) {
console.log('Message received: ' + e.data);
var j = JSON.parse(e.data);
var id = j[0]; // message id
var type = j[1]; // message type (see handlers)
var data = j[2]; // message data (depends on handler)
if (handlers[type]) {
send(id, handlers[type](data));
} else {
console.log('Invalid message: ' + e.type);
}
};
sock.onerror = function(e) {
sock = null;
reconnect();
};
}
function reconnect() {
setTimeout(connect, 1000);
}
function send(id, data) {
var msg = JSON.stringify([id, data || {}]);
sock.send(msg);
}
// define supported message handlers
var handlers = {
'click': click,
'drag': drag,
'mousewheel': mousewheel,
'screenshot': screenshot
};
// generate a mouse click event
function click(data) {
var x = data['x'];
var y = data['y'];
var canvas = $('canvas');
var offset = canvas.offset();
x = offset.left + x;
y = offset.top + y;
var evt = document.createEvent('MouseEvent');
evt.initMouseEvent('mousedown', true, true, window, 1, 0, 0, x, y, false, false, false, false, 0, null);
canvas[0].dispatchEvent(evt);
evt = document.createEvent('MouseEvent');
evt.initMouseEvent('mouseup', true, true, window, 1, 0, 0, x, y, false, false, false, false, 0, null);
canvas[0].dispatchEvent(evt);
}
// generate a mouse drag event
function drag(data) {
var x = data['x'];
var y = data['y'];
var dx = data['dx'];
var dy = data['dy'];
var canvas = $('canvas');
var offset = canvas.offset();
x = offset.left + x;
y = offset.top + y;
dx = x + dx;
dy = y + dy;
var evt = document.createEvent('MouseEvent');
evt.initMouseEvent('mousedown', true, true, window, 1, 0, 0, x, y, false, false, false, false, 0, null);
canvas[0].dispatchEvent(evt);
evt = document.createEvent('MouseEvent');
evt.initMouseEvent('mousemove', true, true, window, 0, 0, 0, dx, dy, false, false, false, false, 0, null);
canvas[0].dispatchEvent(evt);
evt = document.createEvent('MouseEvent');
evt.initMouseEvent('mouseup', true, true, window, 1, 0, 0, dx, dy, false, false, false, false, 0, null);
canvas[0].dispatchEvent(evt);
}
// generate a mouse wheel event
function mousewheel(data) {
var delta = data['delta'];
var canvas = $('canvas');
var evt = document.createEvent('WheelEvent');
evt.initWebKitWheelEvent(0, delta, window, 0, 0, 0, 0, false, false, false, false);
canvas[0].dispatchEvent(evt);
}
// capture a screenshot and send it to the client
function screenshot(data) {
// get the original canvas
var srcCanvas = $('#playn-root canvas');
// create a new canvas
var canvas = $('<canvas />').attr('height', srcCanvas.height() + 'px').attr('width', srcCanvas.width() + 'px')[0];
var context = canvas.getContext('2d');
// copy the original canvas into the new canvas
srcCanvas = srcCanvas[0];
context.drawImage(srcCanvas, 0, 0);
// obtain a png of the canvas
var imageString = canvas.toDataURL();
return {'data': imageString, 'time': new Date().getTime()};
}
// wait for the client to connect
connect();
})();
.ai_button {
border: 1px solid black;
background-color: red;
cursor: pointer;
}
.ai_button:hover {
background-color: yellow;
}
.ai_mark {
position: absolute;
border: 1px solid black;
}
This diff is collapsed.
package ab.demo;
import ab.planner.abTrajectory;
import ab.utils.GameImageRecorder;
import ab.vision.ShowSeg;
/*****************************************************************************
** ANGRYBIRDS AI AGENT FRAMEWORK
** Copyright (c) 2014, XiaoYu (Gary) Ge, Jochen Renz,Stephen Gould,
** Sahan Abeyasinghe,Jim Keys, Andrew Wang, Peng Zhang
** All rights reserved.
**This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
**To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/
*or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.
*****************************************************************************/
public class MainEntry {
// the entry of the software.
public static void main(String args[])
{
String command = "";
if(args.length > 0)
{
command = args[0];
if (args.length == 1 && command.equalsIgnoreCase("-na"))
{
NaiveAgent na = new NaiveAgent();
na.run();
}
else
if(command.equalsIgnoreCase("-cshoot"))
{
ShootingAgent.shoot(args, true);
}
else
if(command.equalsIgnoreCase("-pshoot"))
{
ShootingAgent.shoot(args, false);
}
else
if (args.length == 1 && command.equalsIgnoreCase("-nasc"))
{
ClientNaiveAgent na = new ClientNaiveAgent();
na.run();
}
else
if (args.length == 2 && command.equalsIgnoreCase("-nasc"))
{
ClientNaiveAgent na = new ClientNaiveAgent(args[1]);
na.run();
}
else
if(args.length == 3 && command.equalsIgnoreCase("-nasc"))
{
int id = Integer.parseInt(args[2]);
ClientNaiveAgent na = new ClientNaiveAgent(args[1],id);
na.run();
}
else
if (args.length == 2 && command.equalsIgnoreCase("-na"))
{
NaiveAgent na = new NaiveAgent();
if(! (args[1].equalsIgnoreCase("-showMBR") || args[1].equals("-showReal")))
{
int initialLevel = 1;
try{
initialLevel = Integer.parseInt(args[1]);
}
catch (NumberFormatException e)
{
System.out.println("wrong level number, will use the default one");
}
na.currentLevel = initialLevel;
na.run();
}
else
{
Thread nathre = new Thread(na);
nathre.start();
if(args[1].equalsIgnoreCase("-showReal"))
ShowSeg.useRealshape = true;
Thread thre = new Thread(new ShowSeg());
thre.start();
}
}
else if (args.length == 3 && (args[2].equalsIgnoreCase("-showMBR") || args[2].equalsIgnoreCase("-showReal")) && command.equalsIgnoreCase("-na"))
{
NaiveAgent na = new NaiveAgent();
int initialLevel = 1;
try{
initialLevel = Integer.parseInt(args[1]);
}
catch (NumberFormatException e)
{
System.out.println("wrong level number, will use the default one");
}
na.currentLevel = initialLevel;
Thread nathre = new Thread(na);
nathre.start();
if(args[2].equalsIgnoreCase("-showReal"))
ShowSeg.useRealshape = true;
Thread thre = new Thread(new ShowSeg());
thre.start();
}
else if(command.equalsIgnoreCase("-showMBR"))
{
ShowSeg showseg = new ShowSeg();
showseg.run();
}
else if (command.equalsIgnoreCase("-showReal"))
{
ShowSeg showseg = new ShowSeg();
ShowSeg.useRealshape = true;
showseg.run();
}
else if (command.equalsIgnoreCase("-showTraj"))
{
String[] param = {};
abTrajectory.main(param);
}
else if (command.equalsIgnoreCase("-recordImg"))
{
if(args.length < 2)
System.out.println("please specify the directory");
else
{
String[] param = {args[1]};
GameImageRecorder.main(param);
}
}
else
System.out.println("Please input the correct command");
}
else
System.out.println("Please input the correct command");
}
}
/*****************************************************************************
** ANGRYBIRDS AI AGENT FRAMEWORK
** Copyright (c) 2014, XiaoYu (Gary) Ge, Stephen Gould, Jochen Renz
** Sahan Abeyasinghe,Jim Keys, Andrew Wang, Peng Zhang
** All rights reserved.
**This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
**To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/
*or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.
*****************************************************************************/
package ab.demo;
import java.awt.Point;
import java.awt.Rectangle;
import java.awt.image.BufferedImage;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
import ab.demo.other.ActionRobot;
import ab.demo.other.Shot;
import ab.planner.TrajectoryPlanner;
import ab.utils.StateUtil;
import ab.vision.ABObject;
import ab.vision.GameStateExtractor.GameState;
import ab.vision.Vision;
public class NaiveAgent implements Runnable {
private ActionRobot aRobot;
private Random randomGenerator;
public int currentLevel = 1;
public static int time_limit = 12;
private Map<Integer,Integer> scores = new LinkedHashMap<Integer,Integer>();
TrajectoryPlanner tp;
private boolean firstShot;
private Point prevTarget;
// a standalone implementation of the Naive Agent
public NaiveAgent() {
aRobot = new ActionRobot();
tp = new TrajectoryPlanner();
prevTarget = null;
firstShot = true;
randomGenerator = new Random();
// --- go to the Poached Eggs episode level selection page ---
ActionRobot.GoFromMainMenuToLevelSelection();
}
// run the client
public void run() {
aRobot.loadLevel(currentLevel);
while (true) {
GameState state = solve();
if (state == GameState.WON) {
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
int score = StateUtil.getScore(ActionRobot.proxy);
if(!scores.containsKey(currentLevel))
scores.put(currentLevel, score);
else
{
if(scores.get(currentLevel) < score)
scores.put(currentLevel, score);
}
int totalScore = 0;
for(Integer key: scores.keySet()){
totalScore += scores.get(key);
System.out.println(" Level " + key
+ " Score: " + scores.get(key) + " ");
}
System.out.println("Total Score: " + totalScore);
aRobot.loadLevel(++currentLevel);
// make a new trajectory planner whenever a new level is entered
tp = new TrajectoryPlanner();
// first shot on this level, try high shot first
firstShot = true;
} else if (state == GameState.LOST) {
System.out.println("Restart");
aRobot.restartLevel();
} else if (state == GameState.LEVEL_SELECTION) {
System.out
.println("Unexpected level selection page, go to the last current level : "
+ currentLevel);
aRobot.loadLevel(currentLevel);
} else if (state == GameState.MAIN_MENU) {
System.out
.println("Unexpected main menu page, go to the last current level : "
+ currentLevel);
ActionRobot.GoFromMainMenuToLevelSelection();
aRobot.loadLevel(currentLevel);
} else if (state == GameState.EPISODE_MENU) {
System.out
.println("Unexpected episode menu page, go to the last current level : "
+ currentLevel);
ActionRobot.GoFromMainMenuToLevelSelection();
aRobot.loadLevel(currentLevel);
}
}
}
private double distance(Point p1, Point p2) {
return Math
.sqrt((double) ((p1.x - p2.x) * (p1.x - p2.x) + (p1.y - p2.y)
* (p1.y - p2.y)));
}
public GameState solve()
{
// capture Image
BufferedImage screenshot = ActionRobot.doScreenShot();
// process image
Vision vision = new Vision(screenshot);
// find the slingshot
Rectangle sling = vision.findSlingshotMBR();
// confirm the slingshot
while (sling == null && aRobot.getState() == GameState.PLAYING) {
System.out
.println("No slingshot detected. Please remove pop up or zoom out");
ActionRobot.fullyZoomOut();
screenshot = ActionRobot.doScreenShot();
vision = new Vision(screenshot);
sling = vision.findSlingshotMBR();
}
// get all the pigs
List<ABObject> pigs = vision.findPigsMBR();
GameState state = aRobot.getState();
// if there is a sling, then play, otherwise just skip.
if (sling != null) {
if (!pigs.isEmpty()) {
Point releasePoint = null;
Shot shot = new Shot();
int dx,dy;
{
// random pick up a pig
ABObject pig = pigs.get(randomGenerator.nextInt(pigs.size()));
Point _tpt = pig.getCenter();// if the target is very close to before, randomly choose a
// point near it
if (prevTarget != null && distance(prevTarget, _tpt) < 10) {
double _angle = randomGenerator.nextDouble() * Math.PI * 2;
_tpt.x = _tpt.x + (int) (Math.cos(_angle) * 10);
_tpt.y = _tpt.y + (int) (Math.sin(_angle) * 10);
System.out.println("Randomly changing to " + _tpt);
}
prevTarget = new Point(_tpt.x, _tpt.y);
// estimate the trajectory
ArrayList<Point> pts = tp.estimateLaunchPoint(sling, _tpt);
// do a high shot when entering a level to find an accurate velocity
if (firstShot && pts.size() > 1)
{
releasePoint = pts.get(1);
}
else if (pts.size() == 1)
releasePoint = pts.get(0);
else if (pts.size() == 2)
{
// randomly choose between the trajectories, with a 1 in
// 6 chance of choosing the high one
if (randomGenerator.nextInt(6) == 0)
releasePoint = pts.get(1);
else
releasePoint = pts.get(0);
}
else
if(pts.isEmpty())
{
System.out.println("No release point found for the target");
System.out.println("Try a shot with 45 degree");
releasePoint = tp.findReleasePoint(sling, Math