Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
FIRST Game
Analysis1
Who are we?
• Anne Shade– 21st year in FIRST
– Former FIRST Chesapeake Regional Director
– Former 2377 General Manager and Coach
– Former Mechanical and Systems Engineer, Leidos
• Steve Shade– 20th year in FIRST
– FIRST Chesapeake Head Referee
– Former 2377 Lead Engineer and Coach
– Senior Engineer – Modeling and Simulation, JHU Applied Physics Laboratory
2
Common Themes
• Write it down
• Post it up
• Questions
– What do you want to accomplish?
– Who is responsible?
– How are you going to accomplish it?
– Repeat
3
4
Making Decisions
• Every team should have a method to
make decisions
• Popular vote is not always practical
for robot building purposes, often
misses the quiet contributors
• We like Decision Matrices for
comparing ideas
– Voting can be used to break ties
Decision Matrix - Filled In> 3 better than reference < 3 worse than reference = 3 same as reference
Concept 1
4WD Toughbox
2 Motors
Concept 2
6WD Toughbox
4 Motors
Concept 3
6WD
SuperShifter
4 Motors
Concept 4
Holonomic
4 Motors
Concept 5
Mecanum
4 Motors
Criteria Weight
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Weight 4 3 12 2 8 1 4 2 8 2 8
Power 3 3 9 5 15 5 15 1 3 1 3
Traction 4 3 12 4 16 5 20 1 4 2 8
Maneuverability 5 3 15 4 20 4 20 5 25 5 25
Complexity 3 3 9 2 6 2 6 1 3 1 3
Maintainability 5 3 15 3 15 1 5 2 10 2 10
Reliability 5 3 15 4 20 2 10 2 10 2 10
Durability 5 3 15 3 15 2 10 1 5 1 5
Speed 4 3 12 3 12 5 20 4 16 2 8
Cost 2 3 6 2 4 1 2 2 4 2 4
Weighted Total 120 131 112 88 84
5
Session Overview
Assumptions
• Establish team goals
• Create event goals
• Form match goals
Game Analysis
• Understand how to play the game
• Determine how many points is needed to win a match
• Construct a game strategy to meet the point goal
Implementing a Strategy
• Devise scouting plan
• Implement effective coaching
6
PART 1 - ASSUMPTIONS
7
8
Assumptions
• Your team wants to advance to the District Championship and World Championship
– Be a contributing alliance member
– Consistent and improving on-field performance
– A great robot can win matches, but two good robots with great strategy win more frequently
– Consider other ways to earn District Ranking points
Game & Team
Strategy
Robot Requirements
Robot Design
Robot Construction
Robot Testing
Decision Making
Prioritization
Scouting
Coaching
On Field Performance
Big Picture Process
9
10
District Ranking
• District Championship qualification is determined by points your team earns at first two Chesapeake district events plus team age correction
• FIRST Championship qualification includes points earned at Chesapeake District Championship– District championship weighted 3x district events
Qualifying Points
Alliance Selection
Order
Playoff Performance
AwardsEvent
Ranking Points
Chesapeake District Event 1
Chesapeake District Event 2
Chesapeake District
Champ. (3x)
Team Age Correction (1st & 2nd
year teams)
Total District Ranking Points
11
District Ranking
Qualifying Points
Alliance Selection
Order
Playoff Performance
AwardsEvent
Ranking Points
Chesapeake District Event 1
Chesapeake District Event 2
Chesapeake District Champ.
Team Age Correction (1st & 2nd
year teams)
Total District Ranking Points
Must score points to maximize
these categories!
PART 2 – GAME ANALYSIS
12
Game Analysis Process
Evaluate the Game
• Arena
• Scoring Opportunities
• Fouls
• Ranking Process
Estimate Match Scores
• Alliance
• Team
• Qualifying
• Playoffs
Develop Strategies
• Brainstorm methods to achieve the target match score
• WHAT will our robot do?
Select a Strategy
• Decision Matrix to select a strategy
Develop Robot Requirements
• Demands
• Wishes
Design Robot
• Goal - Achieve Strategy
• Robot Requirements act as design guide
• Specify HOW robot will achieve strategy
13
14
Analyzing the Game
• READ THE RULES!!!!!– Goal: Understand the game
• Examine every possible way to score points, no matter how obscure– How much is each object/robot/action worth?
– Are there bonus points?
– Is there a maximum match score?
– Limited number of objects or do objects cycle?
• Examine every possible way to prevent opponents from scoring– Stealing balls (2000), Capping robots (2004)
• Understand the ranking system– Win-loss-tie, loser’s score, own score plus double
the loser’s score, Coop points, bonus raking points
• What are the key Fouls?
The Big Questions
How many points does an alliance need to score to
win a match?
How many points does our team need to score to help our alliance win a match?
15
Strategy Development
Examples• 2011 (55-60 pts/match/team goal)
– 1 Ubertube top row and 2 top row Logos + 4thplace minibot (58 points)
– 1 Ubertube top row and 1 top row Logo + 1st or 2nd place minibot (60 points)
– 1 Ubertube middle row and 2 middle row Logos + 1st place minibot (58 points)
• 2013 (~72 pts/match/team goal)– 3 High Goal Auto Disc + 4 Pyramid Discs + 30
point Climb (68 points)
– 5 High Goal Auto Disc + 3 cycles @ 4 High Goal Discs + 10 point Climb (76 points)
– 3 High Goal Auto Disc + 20 High Goal Discs + 10 point Climb (88 points)
16
17
• 2014 (~165 pts/match goal)– 15 Auto Mobility + 30 Auto Ball + 4 Cycles @ 20 Ball +
10 Truss (165 points)
– 15 Auto Mobility + 30 Auto Ball + 6 Cycles @ 10 Ball + 10 Truss (165 points)
– 15 Auto Mobility + 30 Auto Ball + 3 Cycles @ 40 Ball (165 points)
– 15 Auto Mobility + 0 Auto Ball + 5 Cycles @ 31 Ball (170 points)
• 2016 (~125 pts/match goal + 1 or 2 RP)– 4 Auto Reach + 1 Auto Cross @ 10+ 10 Auto Boulder +
35 Cross + 5 Low Goals @ 2 + 4 High Goals @ 5 + 5 Challenge + 15 Scale (129 points)
– 1 Auto Reach + 2 Auto Cross @ 10 + 5 Auto Boulder + 8 Cross @ 5 + 4 Low Goals + 3 High Goals + 15 Challenge (120 points)
– 3 Auto Cross @ 10 + 5 Auto Boulder + 5 Cross @ 5 + 5 Low Goals + 3 High Goals + 15 Challenge (125 points)
Strategy Development
Examples
18
AUTO TELE-OPPLAYOFF AUTO TELEOP AUTO TELEOP AUTO TELEOP
Cross Baseline 5 5 15 10
Fuel in High Goal 1 0.3333 6.6667 30 180 6 11.667
Fuel in Low Goal 0.333 0.1111 2.667 2.2222
ROTOR 60 40 40 120 80 60 80
READY FOR TAKEOFF 50 150 50
40 kPA 20
4 ROTORS 100
Total
Max Score
575
Base Score Realistic Score
217.666666756.55555556
• 2017 (~150-200 pts/match goal)– 5 Auto Mobility + 60 Auto Gear + 80 Rotors (5 Gear Cycles) +
50 Climb (195 points)
– 5 Auto Mobility + 0 Auto Gear + 120 Rotors (6 Gear Cycles) + 50 Climb (175 points)
– 5 Auto Mobility + 60 Auto Gear + 40 Rotors (4 Gear Cycles) + 50 Climb (155 points)
– 5 Auto Mobility + 60 Auto Gear + 0 Rotors (0 Gear Cycles) + 50 Climb + 40 kPa Fuel (155 points)
Strategy Development
Examples
19
Develop 3-6 Strategies
Select 1 Strategy
Define Robot Requirements
Next Steps
Game Strategy Thoughts
• Your team does not have to score all of the alliance points, just assist the alliance offense– OFFENSIVE strategy should be priority
– Defensive maneuvers should not be primary strategy, but can be incorporated
• Best to drive and do one thing really well, than do many tasks averagely or worse– Need to do the right thing well
• Example (2014) pickup and manipulate the ball well
20
21
Cost-Benefit Analysis
• For each task you must compare the difficulty of accomplishment to the reward for doing so– Balancing easier than scoring (2012)
– Small balls (2001)
– This is where the strategic value vs. coolness factor decision often pops up
• Vision tetra vs. hanging tetras (2005)
• Scoring Fuel vs. Gears (2017)
• The best tasks to perform are those which are relatively easy, yet provide big points– Climbing the rope (2017)
• Remember denying your opponents 10 points is just as good as scoring 10 points (at least in terms of win/loss)– Descoring much easier than scoring (2003)
22
Prioritization
• Two separate lists– Desired robot qualities
• Things like speed, power, agility, center of gravity
– Desired robot functionality• The things you want your robot to be able to do
– Shoot balls, climb pyramid, traverse field
– Also consider necessary versus desired requirements
– Write these down and post so all can see
• At this point you can merge the two lists, and decide on a drive system and functionalities
23
Strategic Design
• Designing and building a cool robot is a lot of fun
– Designing and building a cool robot that does well in competition is even more fun
• Difficult to go through build without a goal
– Success in competition is clear choice• It’s not just about robots but there’s a reason why the
program is called FIRST Robotics COMPETITION
– Secondary objectives: aesthetics, design elegance, coolness, etc.
• Beware of the “cool” factor
– Can be fun, but sacrificing effectiveness hurts your partners
24
Simplicity & Golden Rules
• Golden Rule #1: Always build within your team’s limits– Evaluate your abilities and resources honestly and
realistically
– Limits are defined by manpower, budget, experience
– Avoid building unnecessarily complex functions
– On the other hand, as you get more experienced start cautiously pushing a few boundaries
– Consider building limits/resources during the offseason instead of sacrificing performance
• Golden Rule #2: If a team has 30 units of robot and functions have maximum of 10 units capability, better to have 3 functions at 10/10 instead of 5 at 6/10– More important to do a few tasks well than a bunch of
tasks mediocre
25
Other Tips
• Remember, you have partners. It’s okay to depend on them for certain tasks (How much you leave to them should be decided by the Golden Rules)– However, be careful not to leave too much
in your partner’s hands
• Try to identify the different types of robots that will exist– Go through the different permutations of
alliances• E.g. How would we do paired with type ‘X’,
against type ‘Y’ and type ‘Z’
• What would we do if we had to play ourselves?
Once Built
• Re-evaluate strategy
– How close are you to your initial goals?
• Be realistic with what you can do
– Know the machine limits
• Watch how game is played at other events
– Game play evolves, very different each week
• Keep improving design
– Apply good ideas from what you watched
• Expand horizons
– Style of play is different in other parts of country26
PART 3 - IMPLEMENTING
A STRATEGY
27
28
Competition Execution
• Scouting– Goal is to collect useful data of actual performance
to feed into match strategy
– Many resources available
• Effective Coaching– Communication
• Negotiating and Consensus Building
– Leadership• Visible team representative
– Teamwork with other Coaches during matches• Alliance selection factor
• Robot Maintenance– Checklists, Checklists, Checklists
– Preventive actions more effective than reactive fixes
29
Scouting
• An area that is often neglected by many
teams
– Offers a great opportunity to get a leg up on the
competition
– Excellent way to involve more students in the
competition
• Crucial for two main reasons
– Predict your opponent’s strategy for future
matches
– Essential for alliance picking
• Especially crucial in getting a good second-round pick
30
• Make sure you check out every team at the event
• Take pictures of every robot– Three views (get the team number in the shot)
– This is important to know who you’re talking about during strategy meetings and making a pick list
• Things to look for– Functionalities
– Type of drivetrain• Number of wheels, traction/wheel type, gearing,
motors
– Quality of construction
• Ask smart questions
Pit Scouting
31
Match Scouting
• Watch every match
• Things to keep track of:– Match score
– Points scored by each team
– Scoring attempts and failures
– Fouls / Technical Fouls
– Autonomous modes, starting position
– General strategy and tendencies
– Drivers and human players• How fast do they react after autonomous…
• HP accuracy (shooting games)
• Make sure you capture this data for all teams in the match
32
Match Scouting
• 1 team of at least 3-6 people – depends on game
• Very tiring, some people have a hard time focusing for the entire day
– Rotate team members, allow time for ample breaks
• Forcing people to scout will result in unreliable data
• Make it fun!
– A team with a culture that respects scouting will result in better scouts
33
Information Management
• You need a way to keep track of all the information your team collects
• Pen & Paper– Have standard forms for Team and Match reports
– Fields for all key information mentioned before
– Easy for everyone to use
• Database– Very efficient way of doing things
– Very easy to generate statistics on each team and rank teams by various criteria
– Requires laptops and tablets
– Can pose difficulties synchronizing
• Possibly a combination of both
34
Advanced Scouting
• Results from current and past seasons
– Match scores, awards, seeding, draft positions, playoff results
– Can further analyze data to find patterns• Least-squares scoring estimation, other custom
metrics
• Known as “OPR”
– High correlation between past success and future success
• Ability to watch events before competing
– Strategies and game play changes throughout the season
35
OPR
• Calculated Contribution / OPR– How can I know how well a team has performed
without watching their matches?
– Could look at average score but that only tells part of the story
– Let each team’s contribution be represented by a variable
– For each alliance, let ti + tj + tk = s where s is the amount of points scored by the alliance
– Solve the matrix
– Now you have calculated the average contribution of each team throughout the regional
– How valuable is this data?• Depends on the game!!!
• More options now with event reported data!
36
Alliance Selection
• The entire process is dependent on scouting
• Make a preliminary pick list on Friday/Saturday night
• Review scouting data
– Discuss criteria of ideal partner based on elimination strategy
– Rank teams from 1 through ~28 based on established criteria
• Slightly more than 24 necessary for full eliminations tournament to allow for robot breakdowns on Saturday
• The “Do Not Pick List”– Should you have one or is it excessive?
37
• Tweak the list through Saturday/Sunday matches
• Make sure your alliance captain is level headed enough not go get flustered on the field
• The second pick can be crucial to the success of your alliance– Excellent teams often (usually) get missed in the
first round
• To break up alliance or not to break up alliances?
• Strategies should be different based on selection point
Alliance Selection
38
• Planning and Execution
– The most important part of the
competition
– Good strategy and scouting can allow a
mediocre robot to win the majority of its
matches
– Good strategy and a good robot are an
almost unbeatable combination
Match Strategies
39
• To develop a good set of strategies, you
need to know what you can do
• Analyze and evaluate your robot’s abilities
– Be honest, don’t under or over estimate
– Factor in the abilities of your drivers
• Create a playbook
– Possible match strategies that can be run
– Different strategies for different circumstances
• Defensive, high risk, safe
Pre - Competition
40
• First priority – Win the match!– Not playing to win is no different than throwing a match
– Throwing matches is UNACCEPTABLE
• Develop a plan for each match with your partners– Everyone agrees to the plan or chaos will ensue on the field
– Don’t wait until you are in the queue for your match to do this
– Develop contingencies
• The plan should outline what each robot will do for the entire match– Never mislead your partner about your abilities
• Can’t do something? Make sure they know that
– Make sure strategies are complementary• Don’t try to occupy the same space of the field, leave each other room
• Create time limits on actions. If something is taking too long, you have to move onto the next– Many teams lose matches because they don’t abandon
failed objectives
Match Plans
41
• Never lose sight of the main goal – winning the match
• Watch the entire field, keep track of the score and the robots
• Make all decisions to deviate from the initial strategy
• Keep the drivers aware of what’s going on– Drivers can only watch the robot and the immediate area
– Time remaining in match
– Drivers will come to depend on the coach, don’t leave them hanging
• Watch the referees
• Communicate with the alliance partner’s coach
• Mentor versus student coach debate
Coaching a Match
42
• If you fall behind, don’t panic, calmly re-evaluate and come up with a new plan
• Leave it all on the field
– Give it your all, don’t be afraid of damage
– That being said, don’t take overly dangerous risks
• After the match
– Sit down with the key team members, discuss what went right and what went wrong
– After a couple of matches, you’ll quickly discard and add strategies
– You must adapt to the competition
During a Match
43
• Change things up– Teams with good scouting will notice if you do
the same thing every match
• Don’t be too conservative or too risky– Know your abilities
– Don’t try to do too much in a match
• Learn how long two minutes is– Run your practices with a timer
• Slow and steady wins the race– Spend 5 seconds setting up, as opposed to 30
seconds of doing it over again
Other Tips
44
• Meet with your new alliance and discuss strategy for playoffs– Consider eating lunch together, there isn’t much
time between alliance selection and start of eliminations
• Make sure key players from all three teams know each other
• Find out if any team needs support– Robot issue? Extra batteries?
• Start planning match strategy for the first round
• Be prepared for more (and more targeted) defense
• Good strategy is the only way to beat a technically much superior alliance
Preparing for the Playoffs
45
• Be prepared to be unconventional if
necessary
• Take advantage of extra planning time to
come up with more effective strategies
– It’s too late to change your robot; it’s not too
late to change your strategies
Preparing for the Playoffs
46
• Read the rules!!
• Develop a clear, consistent strategy for how your robot will play the game
• Remember the Golden Rules
• Scouting is the easiest way to make your team more successful at competition
• The role of the coach cannot be understated
• Each FIRST match is like a high-speed game of chess: You need to have a well thought-out plan, but be prepared to counter your opponents’ moves
• Have fun!!
Final Comments
Backup Material
47
Decision Making Process
• Documented process for the team– Often included in a Team Handbook
• Consensus approaches work well– Decision Matrix tool is a favorite
– Quality Functional Deployment / House of Quality (QFD/HOQ) is more thorough but complicated
– Affinity techniques
– Limit the use of voting• Too easy to go with the “cool” option, even if it doesn’t fit
with the team strategy
• At some point there needs to be 1 person empowered to make a decision for the good of the team– Must be willing to make a decision even when
unpopular for the majority48
Decision Matrix
• Establish decision criteria
• Establish criteria weights
• Complete matrix without weights
shown
• Apply weights
• Evaluate results
49
Decision Criteria
• What are the most important criteria
to your game strategy?
• Need about 5 to 10 criterion.
• Examples
– Scorability
– Complexity
– Reliability
– Penalty Risk
50
Decision Criteria
• Rate the importance of each of your
decision criteria.
• Each criteria needs a rating between
1 (little importance) to 5 (high
importance)
51
Complete Decision Matrix
• Use the decision matrix to select the
team’s strategy.
• Start comparing Concepts 2 through 5
to Concept 1.
• Don’t multiply out the matrix until all
raw scores are filled in.
52
Decision Matrix - Filled In> 3 better than reference < 3 worse than reference = 3 same as reference
Concept 1
4WD Toughbox
2 Motors
Concept 2
6WD Toughbox
4 Motors
Concept 3
6WD
SuperShifter
4 Motors
Concept 4
Holonomic
4 Motors
Concept 5
Mecanum
4 Motors
Criteria Weight
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Weight 4 3 2 1 2 2
Power 3 3 5 5 1 1
Traction 4 3 4 5 1 2
Maneuverability 5 3 4 4 5 5
Complexity 3 3 2 2 1 1
Maintainability 5 3 3 1 2 2
Reliability 5 3 4 2 2 2
Durability 5 3 3 2 1 1
Speed 4 3 3 5 4 2
Cost 2 3 2 1 2 2
Weighted Total
53
Decision Matrix - Filled In> 3 better than reference < 3 worse than reference = 3 same as reference
Concept 1
4WD Toughbox
2 Motors
Concept 2
6WD Toughbox
4 Motors
Concept 3
6WD
SuperShifter
4 Motors
Concept 4
Holonomic
4 Motors
Concept 5
Mecanum
4 Motors
Criteria Weight
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Raw
Score
Weight
ed
Score
Weight 4 3 12 2 8 1 4 2 8 2 8
Power 3 3 9 5 15 5 15 1 3 1 3
Traction 4 3 12 4 16 5 20 1 4 2 8
Maneuverability 5 3 15 4 20 4 20 5 25 5 25
Complexity 3 3 9 2 6 2 6 1 3 1 3
Maintainability 5 3 15 3 15 1 5 2 10 2 10
Reliability 5 3 15 4 20 2 10 2 10 2 10
Durability 5 3 15 3 15 2 10 1 5 1 5
Speed 4 3 12 3 12 5 20 4 16 2 8
Cost 2 3 6 2 4 1 2 2 4 2 4
Weighted Total 120 131 112 88 84
54
5 Why’s
The Best Question for Evaluation and
Working Backwards to Find Root Causes…
• Ask Why 5 times?
– Why Did the Robot Stop?
• The Left Side of the Drive Stopped Working
– Why Did the Left Side of the Drive Stop
Working?
• The Chain Broke
– Why Did the Chain Break?
• We Left a Screwdriver in the Robot
• Not Used to Assign Blame!55