Upload
augustus-kelley
View
217
Download
0
Embed Size (px)
Citation preview
ADSA : RegExprs/8 1
241-423 Advanced Data Structures and Algorithms
• Objective– look at programming with regular expressions
(REs) in Java
Semester 2, 2013-2014
8. Regular Expressions (in Java)
ADSA : RegExprs/8 2
Contents• 1. What are REs?
• 2. First Example
• 3. Case Insensitive Matching
• 4. Some Basic Patterns
• 5. Built-in Character Classes
• 6. Sequencies and Alternatives
• 7. Some Boundary Matches
• 8. Grouping
• 9. (Greedy) Quantifiers
• 10. Three Types of Quantifiers
• 11. Capturing Groups
• 12. Escaping Metacharacters
• 13. split() and REs
• 14. Replacing Text
• 15. Look-ahead & Look-behind
• 16. More Information
ADSA : RegExprs/8 3
1. What are Regular Expressions?
• A regular expression (RE) is a pattern used to search through text.
• It either matches the text (or part of it), or fails to match– you can easily extract the matching parts, or
change them
continued
ADSA : RegExprs/8 4
• REs are not easy to use at first– they're like a different programming language
inside Java
• But, REs bring so much power to string manipulation that they are worth the effort.
• Look back at the "Discrete Math" notes on REs and UNIX grep.
ADSA : RegExprs/8 5
2. First Example
• The RE "[a-z]+" matches a sequence of one or more lowercase letters [a-z] means any character from a to z, and
+ means “one or more”
• Use this pattern to search "Now is the time"• it will match ow• if applied repeatedly, it will find is, the, time, then
fail
ADSA : RegExprs/8 6
Codeimport java.util.regex.*; public class RegexTest
{ public static void main(String args[])
{ String pattern = "[a-z]+"; String text = "Now is the time";
Pattern p = Pattern.compile(pattern); Matcher m = p.matcher(text); while (m.find())
System.out.println( text.substring( m.start(), m.end() ) ); }}
Output: owisthetime
ADSA : RegExprs/8 7
Create a Pattern and Matcher
• Compile the pattern Pattern p = Pattern.compile("[a-z]+");
• Create a matcher for the text using the pattern Matcher m = p.matcher("Now is the time");
ADSA : RegExprs/8 8
Finding a Match
• m.find() returns true if the pattern matches any part of the text string; false otherwise
• If called again, m.find() will start searching from where the last match was found.
ADSA : RegExprs/8 9
Printing what was Matched
• After a successful match: – m.start() returns the index of the first character
matched– m.end() returns the index of the last character
matched, plus one
• This is what most String methods require– e.g. "Now is the time".substring(m.start(),
m.end()) returns the matched substring
continued
ADSA : RegExprs/8 10
• If the match fails, m.start() and m.end() throw an IllegalStateException– this is a RuntimeException, so you don’t have to catch
it
ADSA : RegExprs/8 11
Test Rigpublic class TestRegex
{
public static void main(String[] args)
{ if (args.length != 2) {
System.out.println("Usage: java TestRegex string regExp");
System.exit(0);
}
System.out.println("Input: \"" + args[0] + "\"");
System.out.println("Regular expression: \"" + args[1] + "\"");
Pattern p = Pattern.compile(args[1]);
Matcher m = p.matcher(args[0]);
while (m.find())
System.out.println("Match \"" + m.group() + "\" at positions "+
m.start() + "-" + (m.end()-1));
} // end of main()
} // end of TestRegex class
ADSA : RegExprs/8 12
• m.group() returns the string matched by the pattern– usually used instead of String.substring()
ADSA : RegExprs/8 14
3. Case Insensitive Matching
String sentence = "The quick brown fox and BROWN tiger jumps over the lazy dog";
Pattern pattern = Pattern.compile("brown",
Pattern.CASE_INSENSITIVE);
Matcher matcher = pattern.matcher(sentence);
while (matcher.find())
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(), matcher.end());
Text "brown" found at 10 to 15.Text "BROWN" found at 24 to 29.
a flag
ADSA : RegExprs/8 15
• Many flags can also be written as part of the RE:Pattern pattern = Pattern.compile( "(?i)brown" );
ADSA : RegExprs/8 16
4. Some Basic Patternsabc exactly this sequence of three letters
[abc] any one of the letters a, b, or c
[^abc] any character except one of the letters a, b, or c
[a-z] any one character from a through z
[a-zA-Z0-9] any one letter or digit
• The set of characters defined by [...] is called a character class.
ADSA : RegExprs/8 17
Example
// search for a string that begins with "bat" and a number in the range [3-7]
String input =
"bat1, bat2, bat3, bat4, bat5, bat6, bat7, bat8";
Pattern pattern = Pattern.compile( "bat[3-7]" );
Matcher matcher = pattern.matcher(input);
while (matcher.find())
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(), matcher.end());
Text "bat3" found at 12 to 16.Text "bat4" found at 18 to 22.Text "bat5" found at 24 to 28.Text "bat6" found at 30 to 34.Text "bat7" found at 36 to 40.
ADSA : RegExprs/8 18
5. Built-in Character Classes. any one character except a line terminator
\d a digit: [0-9]
\Da non-digit: [^0-9]
\s a whitespace character: [ \t\n\x0B\f\r]
\S a non-whitespace character: [^\s]
\w a word character: [a-zA-Z_0-9]
\W a non-word character: [^\w]
Notice the space
continued
ADSA : RegExprs/8 19
• In Java you will need to "double escape" the RE backslashes:
\\d \\D \\S \\s \\W \\w
when you use them inside Java strings
• Note: if you read in a pattern from somewhere (the keyboard, a file), there's no need to double escape the text.
ADSA : RegExprs/8 20
Example 1
// search for a whitespace, 'f', and any two chars
Pattern pattern = Pattern.compile( "\\sf.." );
Matcher matcher = pattern.matcher(
"The quick brown fox jumps over the lazy dog");
while (matcher.find()) {
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(), matcher.end());
Text " fox" found at 15 to 19.
ADSA : RegExprs/8 21
Example 2
// match against a digit followed by a wordPattern p = Pattern.compile( "\\d+\\w+" );Matcher m = p.matcher("this is the 1st test string");
if(m.find()) System.out.println("matched [" + m.group() +
"] from " + m.start() + " to " + m.end() );else System.out.println("didn’t match");
matched [1st] from 12 to 15
ADSA : RegExprs/8 22
Subtraction
• You can use subtraction with character classes. – e.g. a character class that matches everything
from a to z, except the vowels (a, e, i, o, u)– written as [a-z&&[^aeiou]]
ADSA : RegExprs/8 23
Search excluding vowels
Pattern pattern = Pattern.compile( "[a-z&&[^aeiou]]" );
Matcher matcher = pattern.matcher("The quick brown fox.");
while (matcher.find()) {
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(), matcher.end());
Text "h" found at 1 to 2.Text "q" found at 4 to 5.Text "c" found at 7 to 8.Text "k" found at 8 to 9.Text "b" found at 10 to 11.Text "r" found at 11 to 12.Text "w" found at 13 to 14.Text "n" found at 14 to 15.Text "f" found at 16 to 17.Text "x" found at 18 to 19.
ADSA : RegExprs/8 24
6. Sequences and Alternatives
• Two patterns matches in sequence:– e.g., [A-Za-z]+[0-9] will match one or more
letters immediately followed by one digit
• The bar, |, is used to separate alternatives– e.g., abc|xyz will match either abc or xyz– best to use brackets to make the scope clearer
• (abc)|(xyz)
ADSA : RegExprs/8 25
Search for 't' or 'T'
Pattern pattern = Pattern.compile( "[t|T]" );
Matcher matcher = pattern.matcher(
"The quick brown fox jumps over the lazy dog");
while (matcher.find())
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(), matcher.end());
Text "T" found at 0 to 1.Text "t" found at 31 to 32.
ADSA : RegExprs/8 26
7. Some Boundary Matchers
^ the beginning of a line
$ the end of a line
\b a word boundary
\B not a word boundary
\G the end of the previous match
written as \\b, \\B, and \\G in Java strings
ADSA : RegExprs/8 27
Find "dog" at End of Line
Pattern pattern = Pattern.compile( "dog$" );
Matcher matcher = pattern.matcher(
"The quick brown dog jumps over the lazy dog");
while (matcher.find())
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(), matcher.end());
Text "dog" found at 40 to 43.
ADSA : RegExprs/8 28
Look for a Country
ArrayList<String> countries = new ArrayList<String>();
countries.add("Austria"); : // more adds
/* Look for a country that starts with "I" with any 2nd letter and either "a" or "e" in the 3rd position. */
Pattern pattern = Pattern.compile( "^I.[ae]" );
for (String c : countries) {
Matcher matcher = pattern.matcher(c);
if (matcher.lookingAt())
System.out.println("Found: " + c);
}
Found: IcelandFound: IraqFound: IrelandFound: Italy
continued
ADSA : RegExprs/8 29
• m.lookingAt() returns true if the pattern matches at the beginning of the text string, false otherwise.
ADSA : RegExprs/8 30
Word Boundaries: \b \B
• A word boundary is a position between \w and \W (non-word char), or at the beginning or end of a string.
• A word boundary is zero length.
ADSA : RegExprs/8 31
Examples String s = "A nonword boundary is the opposite of a word boundary, " +
"i.e., anything other than a word boundary.";
// match all words "word"
Pattern p1 = Pattern.compile("\\bword\\b");
Matcher m1 = p1.matcher(s);
while (m1.find())
System.out.println("p1 match: " + m1.group() + " at " + m1.start());
// match word ending with "word" but not the word "word"
Pattern p2 = Pattern.compile("\\Bword\\b");
Matcher m2 = p2.matcher(s);
while (m2.find())
System.out.println("p2 match: " + m2.group() + " at " + m2.start());
p1 match: word at 40p1 match: word at 83
p2 match: word at 5
ADSA : RegExprs/8 32
8. Grouping
• A group treats multiple characters as a single unit. – a group is created by placing characters inside
parentheses– e.g. the RE (dog) is the group containing the
letters "d" "o" and "g".
ADSA : RegExprs/8 33
Find the Words 'the' or 'quick'
String text = "the quick brown fox jumps over the lazy dog";
Pattern pattern = Pattern.compile( "(the)|(quick)" );
Matcher matcher = pattern.matcher(text);
while (matcher.find()) {
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(), matcher.end());
Text "the" found at 0 to 3.Text "quick" found at 4 to 9.Text "the" found at 31 to 34.
ADSA : RegExprs/8 34
9. (Greedy) QuantifiersX represents some pattern:
X? optional, X occurs once or not at all
X* X occurs zero or more times
X+ X occurs one or more times
X{n} X occurs exactly n times
X{n,} X occurs n or more times
X{n,m} X occurs at least n but not more than m times
ADSA : RegExprs/8 35
Example
String[] exprs = {
"x?", "x*", "x+", "x{2}", "x{2,}", "x{2,5}" };
String input = "xxxxxx yyyxxxxxx zzzxxxxxx";
for (String expr : exprs) {
Pattern pattern = Pattern.compile(expr);
Matcher matcher = pattern.matcher(input);
System.out.println("--------------------------");
System.out.format("regex: %s %n", expr);
while (matcher.find())
System.out.format("Text \"%s\" found at %d to %d.%n",
matcher.group(), matcher.start(),matcher.end());
ADSA : RegExprs/8 36
Output
Regex: x?
Text "x" found at 0 to 1.
Text "x" found at 1 to 2.
Text "x" found at 2 to 3.
Text "x" found at 3 to 4.
Text "x" found at 4 to 5.
Text "x" found at 5 to 6.
Text "" found at 6 to 6.
Text "" found at 7 to 7.
Text "" found at 8 to 8.
Text "" found at 9 to 9.
Text "x" found at 10 to 11.
Text "x" found at 11 to 12.
Text "x" found at 12 to 13.
Text "x" found at 13 to 14.
Text "x" found at 14 to 15.
Text "x" found at 15 to 16.
Text "" found at 16 to 16.
Text "" found at 17 to 17.
Text "" found at 18 to 18.
Text "" found at 19 to 19.
Text "x" found at 20 to 21.
Text "x" found at 21 to 22.
Text "x" found at 22 to 23.
Text "x" found at 23 to 24.
Text "x" found at 24 to 25.
Text "x" found at 25 to 26.
Text "" found at 26 to 26.
------------------------------
continued
ADSA : RegExprs/8 37
Regex: x*
Text "xxxxxx" found at 0 to 6.
Text "" found at 6 to 6.
Text "" found at 7 to 7.
Text "" found at 8 to 8.
Text "" found at 9 to 9.
Text "xxxxxx" found at 10 to 16.
Text "" found at 16 to 16.
Text "" found at 17 to 17.
Text "" found at 18 to 18.
Text "" found at 19 to 19.
Text "xxxxxx" found at 20 to 26.
Text "" found at 26 to 26.
------------------------------
Regex: x+
Text "xxxxxx" found at 0 to 6.
Text "xxxxxx" found at 10 to 16.
Text "xxxxxx" found at 20 to 26.
------------------------------
Regex: x{2}
Text "xx" found at 0 to 2.
Text "xx" found at 2 to 4.
Text "xx" found at 4 to 6.
Text "xx" found at 10 to 12.
Text "xx" found at 12 to 14.
Text "xx" found at 14 to 16.
Text "xx" found at 20 to 22.
Text "xx" found at 22 to 24.
Text "xx" found at 24 to 26.
------------------------------
Regex: x{2,}
Text "xxxxxx" found at 0 to 6.
Text "xxxxxx" found at 10 to 16.
Text "xxxxxx" found at 20 to 26.
------------------------------
Regex: x{2,5}
Text "xxxxx" found at 0 to 5.
Text "xxxxx" found at 10 to 15.
Text "xxxxx" found at 20 to 25.
ADSA : RegExprs/8 38
Matching SSN Numbers
ArrayList<String> input = new ArrayList<String>();
input.add("123-45-6789");
input.add("9876-5-4321");
input.add("987-65-4321 (attack)");
input.add("987-65-4321 ");
input.add("192-83-7465");
for (String ssn : input)
if (ssn.matches( "^(\\d{3}-?\\d{2}-?\\d{4})$" ))
System.out.println("Found good SSN: " + ssn);
Found good SSN: 123-45-6789Found good SSN: 192-83-7465
continued
ADSA : RegExprs/8 39
• String.matches(String regex) returns true or false depending on whether the string matches the RE (regex).
• str.matches(regex) is the same as:Pattern.matches(regex, str)
ADSA : RegExprs/8 40
10. Three Types of Quantifiers
• 1. A greedy quantifier will match as much as it can, and back off if it needs to– see examples on previous slides
• 2. A reluctant quantifier will match as little as possible, then take more if it needs to– you make a quantifier reluctant by adding a ?:X?? X*? X+? X{n}? X{n,}? X{n,m}?
continued
ADSA : RegExprs/8 41
• 3. A possessive quantifier will match as much as it can, and never lets go– you make a quantifier possessive by appending a
+:X?+ X*+ X++ X{n}+ X{n,}+ X{n,m}+
ADSA : RegExprs/8 42
Quantifier Examples
• The text is "aardvark".
• 1. Use the pattern a*ardvark (a* is greedy)– the a* will first match aa, but then ardvark
won’t match
– the a* then “backs off” and matches only a single a, allowing the rest of the pattern (ardvark) to succeed continued
ADSA : RegExprs/8 43
• 2. Use the pattern a*?ardvark (a*? is reluctant)– the a*? will first match zero characters (the null
string), but then ardvark won’t match
– the a*? then extends and matches the first a, allowing the rest of the pattern (ardvark) to succeed
continued
ADSA : RegExprs/8 44
• 3. Using the pattern a*+ardvark (a*+ is possessive)– the a*+ will match the aa, and will not back
off, so ardvark never matches and the pattern match fails
ADSA : RegExprs/8 45
Reluctant Example
Pattern pat = Pattern.compile( "e.+?d" );
Matcher mat = pat.matcher("extend cup end table");
while (mat.find())
System.out.println("Match: " + mat.group());
Output:Match: extendMatch: end
ADSA : RegExprs/8 46
11. Capturing Groups
• Parentheses are used for grouping, but they also capture (keep for later use) anything matched by that part of the pattern.
• Example: ([a-zA-Z]*)([0-9]*) matches any number of letters followed by any number of digits
• If the match succeeds:– \1 holds the matched letters– \2 holds the matched digits– \0 holds everything matched by the entire pattern
continued
ADSA : RegExprs/8 47
• Capturing groups are numbered by counting their opening parentheses from left to right:– ( ( A ) ( B ( C ) ) )
1 2 3 4\0 = \1 = ((A)(B(C))), \2 = (A), \3 = (B(C)), \4 = (C)
• Example: ([a-zA-Z])\1 will match a double letter, such as letter
continued
ADSA : RegExprs/8 48
• A word puzzle: "what is the only word in English which has three consecutive double letters?"
• Two possible answers are "sweet-tooth" and "hoof-footed", but they use hyphens, which I'm not allowing
ADSA : RegExprs/8 49
Matcher.group()
• If m is a matcher that has just got a successful match, then– m.group(n) returns the String matched by
capturing group n• this could be an empty string• this will be null if the pattern as a whole matched
but this particular group didn’t match anything
– m.group(0) returns the String matched by the entire pattern (same as m.group())• this could be an empty string
ADSA : RegExprs/8 50
Examples
• Move all the consonants at the beginning of a string to the end – "sheila" becomes "eilash"
Pattern p = Pattern.compile( "([^aeiou]*)(.*)" );Matcher m = p.matcher("sheila");if (m.matches()) System.out.println(m.group(2) + m.group(1));
• (.*) means “all the rest of the chars”
ADSA : RegExprs/8 51
12. Escaping Metacharacters
• A lot of special characters – parentheses, brackets, braces, stars, the plus sign, etc. – are used in REs– they are called metacharacters
continued
ADSA : RegExprs/8 52
• Suppose you want to search for the character sequence a* (an a followed by an ordinary "*")– "a*"; doesn’t work; that means “zero or more a's”
– "a\*"; doesn’t work; since a star doesn’t need to be escaped in Java String constants; Java ignores the \
– "a\\*" does work; it’s the three-char string a, \, *
• Just to make things even more difficult, it’s illegal to escape a non-metacharacter in a RE.
ADSA : RegExprs/8 53
13. split() and REs
String colours = "Red,White, Blue Green Yellow, Orange";
// Pattern for finding commas and whitespaces
Pattern splitter = Pattern.compile( "[,\\s]+" );
String[] cols = splitter.split(colours);
for (String colour : cols)
System.out.println("Colour = \"" + colour + "\"");
continued
ADSA : RegExprs/8 54
• Or use String.split(String regex):
String colours = "Red,White, Blue Green Yellow, Orange";
// Pattern for finding commas and whitespaces
String[] cols = colours.split( "[,\\s]+" );
for (String colour : cols)
System.out.println("Colour = \"" + colour + "\"");
ADSA : RegExprs/8 55
14. Replacing Text
• If m is a matcher, then– m.replaceFirst(replacement) returns a new
String where the first substring matched by the pattern is replaced by replacement
– m.replaceAll(replacement) returns a new String where all matched substrings are replaced
ADSA : RegExprs/8 56
Example 1
Pattern pattern = Pattern.compile( "a" );
Matcher matcher = pattern.matcher("a b c a b c");
String output = matcher.replaceAll("x"); // is "x b c x b c"
ADSA : RegExprs/8 57
Example 2
String str = "Java1 Java2 JDK Java2S Java2s.com"; Pattern pat = Pattern.compile( "Java.*? " ); Matcher mat = pat.matcher(str); System.out.println("Original: " + str); str = mat.replaceAll("Java "); System.out.println("Modified: " + str);
Original : Java1 Java2 JDK Java2S Java2s.comModified : Java Java JDK Java Java2s.com
ADSA : RegExprs/8 58
15. Look-ahead & Look-behind
• A Look-ahead expression looks forward, starting from its location in the pattern, continuing to the end of the input.
• A Look-behind expression starts at the beginning of the pattern and continues up to the look-behind expression.
• These patterns do not capture values.
ADSA : RegExprs/8 59
Operations
• (?:X) X, as a non-capturing group
• (?=X) X, via zero-width positive look-ahead
• (?!X) X, via zero-width negative look-ahead
• (?<=X) X, via zero-width positive look-behind
• (?<!X) X, via zero-width negative look-behind
• (?<X) X, as an independent, non-capturing group
ADSA : RegExprs/8 60
Look-ahead Example 1
• Does the input text contain “incident” but not “theft” anywhere.
• Pattern: "(?!.*theft).*incident.*"
• Result:– "There was a crime incident" matches– "The incident involved a theft" no match– "The theft was a serious incident" no match
ADSA : RegExprs/8 61
Example 2
String regex = "John (?!Smith)[A-Z]\\w+";
Pattern pattern = Pattern.compile(regex);
String str = "I think that John Smith is a fictional character. His real name might be John Jackson, John Gestling, or John Hulmes for all we know.";
Matcher matcher = pattern.matcher(str);
while (matcher.find())
System.out.println("MATCH: " + matcher.group());
John names excluding John Smith
MATCH: John JacksonMATCH: John GestlingMATCH: John Hulmes
ADSA : RegExprs/8 62
Look-behind Example
// find text which is preceded by "http://"
Pattern pat = Pattern.compile( "(?<=http://)\\S+" );
String str = "The Java2s website can be found at http://www.java2s.com. There, you can find some Java examples.";
Matcher matcher = pat.matcher(str);
while (matcher.find())
System.out.println(":" + matcher.group() + ":");
:www.java2s.com.:
ADSA : RegExprs/8 63
16. More Information
• Look in any Java textbook that deals with J2SE 1.4 or later.– I've placed a RE extract from "Java: How to
Program", 7th ed. on the ADSA website
• I explained REs in the "Discrete Maths" subject (using grep).
continued
ADSA : RegExprs/8 64
• The Java tutorial on REs is very good:– http://java.sun.com/docs/books/tutorial/
essential/regex/
• Online tutorials:– http://ocpsoft.com/opensource/
guide-to-regular-expressions-in-java-part-1/– and part-2
continued