Embed Size (px)
Citation preview
Apple’s Bug ReportMacIT 894
Mike Boylan Systems Engineer
Pittsburgh, PA
MacIT 894
Who am I?• Systems engineer for
Robert Morris University in Pittsburgh, PA
• Core University infrastructure, telephony, “The Mac guy”
• @mboylan on twitter • Pittsburgh politics & Volvo
MacIT 894
Session Disclaimer“Session presenter does not work for Apple, and although the practice of filing thorough bugs with Apple has benefited his organization immensely, he cannot guarantee the same for any other company or organization. Responses to bugs and impact assessments are handled at Apple’s discretion on a bug-by-bug basis.”
MacIT 894
Software has bugs
MacIT 894
How to identify
Does the behavior you’re seeing seem unexpected and/or odd?
Is the behavior you’re seeing different from that in previous versions?
Are you able to repeat the behavior under a particular set of circumstances?
MacIT 894
When you notice a bug in an Apple product, you should file it.
MacIT 894
Why you should file
Bug reports provide a direct communication path to Apple product engineering. Bug reports allow Apple to help you*.
!
*But should not be used in lieu of AppleCare enterprise support
MacIT 894
But should I file...?
MacIT 894
Should I file a bug...
For “trivial” issues? Yes!
For an issue that only happened once? Maybe!
For something that seems obvious? Yes!
For something that someone else filed? Yes!
MacIT 894
“If an issue does not exist in bug reporter, it does not exist.”
Apple Engineering Mantra
!
Source: WWDC 2013 “Maximizing Apple Development Resources” session
MacIT 894
How?
MacIT 894
http://bugreport.apple.comYou must be a registered Apple developer at any tier to login (can be free).
MacIT 894
Other fileable itemsFeature/enhancement requests
Requests for updates to, or expansions to, product documentation
Security exploits and vulnerabilities
User interface or usability related items
Power and/or performance issues
MacIT 894
Let’s go on a tour!
MacIT 894
MacIT 894
Available categories
MacIT 894
MacIT 894
You can even file bugs about bug reporter from within bug reporter...
MacIT 894
Bug reporting basics
MacIT 894
Bug reporting basicsUse a descriptive title
Be thorough, precise, and accurate
Include as much information as possible (logs, screenshots, etc.)
File one report for one issue
File as early as possible
MacIT 894
File as early as possible
MacIT 894
File as early as possibleFi
x Pr
obab
ility
Seed 1 Seed 2 ... Last Seed GM Seed 1 Seed 2 ...
Current ReleaseFuture Release
MacIT 894
Components of an Apple bug report
MacIT 894
Report Title
Problem Details
Reproducibility
Configuration Information
Additional Enclosures
Report components
MacIT 894
Report Title
Problem Details
Reproducibility
Configuration Information
Additional Enclosures
Report components
MacIT 894
Report Title
“Sells” your issue to Apple engineering
Be clear and succinct
Should stand on its own
Bonus points for including product name and version number(s)
MacIT 894
Report Title
Sample titles:
iOS 7 | Siri & Navigation | Beginning navigation while locked doesn't override screen sleepiOS 7 | Siri | Location information shows wrong distance
MacIT 894
Report Title
Problem Details
Reproducibility
Configuration Information
Additional Enclosures
Report components
MacIT 894
Description
Steps to Reproduce
Expected Results
Actual Results
Regression?
Problem Details
MacIT 894
Description
Steps to Reproduce
Expected Results
Actual Results
Regression?
Problem Details
MacIT 894
DescriptionExpand on your title
Tell Apple what is wrong, in summary format (but be thorough and descriptive)
Why is this a problem?
Include exact error message text
Limited to 3,000 characters here
MacIT 894
Description
Example (15073676):
When attempting to save the Installer Log from the Installer Log window in the OS Installer, the save window defaults to root's home folder. When attempting to save there, a message is displayed that "You don't have permission to save the file at that location."
MacIT 894
Description
Steps to Reproduce
Expected Results
Actual Results
Regression?
Problem Details
MacIT 894
Steps to Reproduce
Assume Apple knows nothing about issue
Use numbered list to describe step-by-step
Attempt to anticipate questions
Don’t leave out anything -- Everything and anything could be important and relevant
MacIT 894
DescriptionExample (15073676):
1.) Download OS X Mavericks from the Mac App Store
2.) Boot into the installer and begin installing OS X Mavericks
3.) Open the Installer Log
4.) Wait for the install to finish
5.) Press the button to Save the log
6.) Observe that the location defaults to root's home folder
7.) Try to save the file
8.) Observe the error message that is attached. "You don't have permission to save the file at that location."
MacIT 894
Description ImprovedExample (15073676):
1.) Download OS X Mavericks from the Mac App Store
2.) Boot into the installer and begin installing OS X Mavericks <- How?
3.) Open the Installer Log <- How? From where?
4.) Wait for the install to finish
5.) Press the button to Save the log <- What button? Where?
6.) Observe that the location defaults to root's home folder
7.) Try to save the file <- How?
8.) Observe the error message that is attached. "You don't have permission to save the file at that location."
MacIT 894
Use a ScreenshotExample (15073676):
!
!
!
!
MacIT 894
Description
Steps to Reproduce
Expected Results
Actual Results
Regression?
Problem Details
MacIT 894
Expected vs Actual Results
What do you think should have happened?
Why do you think x should have happened?
Critical piece of information
Apple may disagree with you
MacIT 894
Expected vs Actual Results
Example (15073676):
Expected Results:
The Save dialog would default to somewhere where the permissions allow writing, such as the root of the soon to be boot volume.
!
Actual Results:
The save dialog defaults to root's home directory, in which the OS Installer environment is unable to write.
MacIT 894
Description
Steps to Reproduce
Expected Results
Actual Results
Regression?
Problem Details
MacIT 894
Regression
Is this something that worked correctly in a previous release?
Include previous version numbers/builds where behavior worked as expected
Include first known version/build number where problem began occurring
MacIT 894
Report Title
Problem Details
Reproducibility
Configuration Information
Additional Enclosures
Report components
MacIT 894
ReproducibilityAlways
Sometimes
Rarely
Unable
I didn’t Try
Not Applicable
} How often can you duplicate the issue?
MacIT 894
ReproducibilityAlways
Sometimes
Rarely
Unable
I didn’t Try
Not Applicable
}}
How often can you duplicate the issue?
Don’t use this
MacIT 894
ReproducibilityAlways
Sometimes
Rarely
Unable
I didn’t Try
Not Applicable
}}}
How often can you duplicate the issue?
Don’t use this
Feature requests
MacIT 894
Report Title
Problem Details
Reproducibility
Configuration Information
Additional Enclosures
Report components
MacIT 894
Configuration Info
Hardware model and version
iOS/OS X version and build
Firmware version
Accessory information (wireless, car, etc.)
Non-default settings/changes
MacIT 894
Report Title
Problem Details
Reproducibility
Configuration Information
Additional Enclosures
Report components
MacIT 894
Additional Enclosures
Give Apple everything you can!
!
Lots of details: https://developer.apple.com/bug-reporting
MacIT 894
OS X EnclosuresAlways include: sysdiagnose, serverloggather, screenshots, video screen captures
Profiles/MCX: sudo defaults write /Library/Preferences/com.apple.MCXDebug debugOutput -2
sudo defaults write /Library/Preferences/com.apple.MCXDebug collateLogs 1
Directory services: sudo odutil set log debug
WiFi: sudo /usr/libexec/airportd debug +AllUserland +AllDriver +LogFile
Profile Manager (for Server): sudo defaults write /Library/Preferences/com.apple.ProfileManager debugMode 3
MacIT 894
iOS Enclosures
Crash Logs: Connect your device, sync your device in iTunes ~/Library/Logs/CrashReporter/MobileDevice/[Device Name]
Console Logs: Connect your device, open Xcode organizer, [your device], Console, Save Log as...
Screenshots: Press and release home and power buttons at same time. Saved to camera roll.
Stackshot logs (performance issues): Press and release one of the volume buttons and home button at same time~/Library/Logs/CrashReporter/MobileDevice/[Device Name]/stacks*
MacIT 894
What happens after I file a bug with Apple?
MacIT 894
“Every time I file a bug with Apple, it just sits there stagnant with no reply.”
MacIT 894
This is good. You provided enough information. Hopefully it will be fixed.
MacIT 894
“Every time I file a bug with Apple, it gets marked as a duplicate.”
MacIT 894
This is still good. Apple uses the # of duplicates as a seriousness indicator.
MacIT 894
“Apple e-mailed me back asking for more information.”
MacIT 894
This is also good. You might learn of some new debug logging options.
MacIT 894
Reply to Apple as soon as possible to help ensure they don’t run out of time.
MacIT 894
“Apple e-mailed me back saying they’re not going to fix my bug.”
MacIT 894
Sorry :(
MacIT 894
BUG
MacIT 894
Sufficient information
!(No reply)
Duplicate !!
(Reply)
Insufficient information
!(Reply)
BUG
Not to be fixed !
(Reply)
MacIT 894
Recap on Bugs
MacIT 894
RecapAssume Apple knows nothing about issue
File as early as possible & be responsive
Stick to format provided in bug reporter
Be succinct, precise, and accurate
Be prepared to use other resources for urgent issues impacting your business
MacIT 894
Two Recent Examples
MacIT 894
Two-Sided Printing
MacIT 894
10.8
MacIT 894
10.9
MacIT 894
“Engineering has determined that this issue behaves as intended based on the following: !
There is a comment in the code: // check if any 3rd party PDE handles Duplex - if one // is found, send a notification to hide our Duplex UI !
and it is related to Bug ID 4827573: EPSON's CUPS Driver can not Disable Duplex setting of Layout PDE. !
This is done by design.”
MacIT 894
Mavericks Default File Sharing Protocol
MacIT 894
MacIT 894
“When you connect from a Mac using OS X Mavericks to another computer using file sharing, your Mac automatically tries to use SMB to communicate. If SMB is not available, Mavericks tries to connect using Apple File Protocol (AFP).” !
HT5884
MacIT 894
Go > Connect to Server = AFP !
Finder Sidebar = SMB
MacIT 894
Engineering has provided the following: !
Choosing Go > Network and then connecting via Finder also uses SMB. This is be a bug [sic] in Finder's Connect To. You can override Connect To by typing smb:// in the URL as well. !
We have filed an internal bug report, Bug ID 16205278. We provide this ID so you can inquire regarding status of the issue in the future. !
It's a bug that will be fixed, so we won't be updating the documentation. We are now closing this bug report.
MacIT 894
One more thing...
MacIT 894
“Submit Feedback” &
“Was this helpful?”
MacIT 894
A new menu option in most new Apple apps that either opens your browser to a feedback page, or opens your mail client.
MacIT 894
MacIT 894
Example direct e-mail
MacIT 894
Example web feedback form
MacIT 894
Your feedback goes directly to an internal mailing list at Apple where real engineers and managers read it. You most likely won’t get a response.
MacIT 894
From within OS X Help…
MacIT 894
MacIT 894
MacIT 894
On kbase articles…
MacIT 894
MacIT 894
New VPP is one great example of Apple listening to customer feedback. !
Also, new Device Enrollment Program.
