Internet and WWW Services

• Security• Types of Services• Vended versus Internally Provided• Costs and Benefits• Servers and Clients• Potential Problems• Stats

General Network Security

• Isolated Servers• Restricted Subnets• Firewalls• Proxy Servers

WWW Application Security

• OS Level• Server Level• Program Level

Types of WWW Services

• Static Data• Server Search Engines• Dynamic Data• Server Applications• Java Enabled

Vended

• Which Vendor

• How Much Do They Do

– HTML

– Graphics

– Design & Layout

– Programming

• Bandwidth

– Total

– Dedicated

Internally Provided WWW Server

• For who?• How many services, how much traffic?• For what use (scope the server) ?

Cost of a WWW Service

• Server Usage• Disk Space• Network Bandwidth• Router or LAN Load• Application Development with Limited Capabilities• Application Development with Limited

Standardization

Benefits

• High-touch, High Impact Narrow-casting• Kiosks• Fast, Simple Apps From Central Server• Built-in Protocols• Potentially Large Installed Client Base

Shopping List

• Server Machine and O/S• Network Access• WWW Server• WWW Client• Server Programming Tools• Data and/or Databases

Which Server Platform?

• Unix• NT

Which Server?

• CREN• Microsoft• Netscape - Communication or Commerce• O’Reilly• WebForce• Oracle WebServer

Client Compliance Level

• HTML 2.0• HTML 3.0• Netscape Enhancements• Java• Lynx (Text Browser)

CGI-BIN Risks

• Dangerous Programs or Scripts• User-supplied Programs or Scripts

Robots and Other Network Creatures

• Problems with “Automated Agents”• Deterring Robots• Reacting to Robots

WWW Server Stats

JanuaryFebruary

Outside

100000

150000

200000

250000

300000

350000

400000

WWW Accesses per Week

UNM35%

Outside65%

Outside

100,000

200,000

300,000

400,000

500,000

600,000

January February March

WWW Server Stats

JanuaryFebruary

Outside

100000

150000

200000

250000

300000

350000

400000

UNM35%

Outside65%

Web Mining

Web based information extraction

Why the Web(web = web browser)

• Ubiquitous: – Web browsers are on every desktop, every PC, Mac,

workstation, and terminal.

• Platform independence– Use of Java and server side programs means clicking on a

button does the same thing everywhere.

Natural Language

News ServicesNews Services

Multidimensional vectors

Markov objectsID3ID3

Word frequency

Data warehouse

Data CleansingData Compression

Text Mining

Factorial Analysis

Keyword Search

Decision Trees

Tri-Grams

Tri-Letter Sets

Hidden Information

DATA CleansedData

ExtractedData

DisplayResults

What Kind of Data?

• Usenet News– Most places have Multi gigs of news

• System accounting files – Can tell who is doing what, when

• Misc. Web pages– A variety of interesting information

• Listserver or public system email– We keep email concerning system problems

Cleansing Data

• News article– NNTP fields

– signatures

• Web Page– HTML codes

– descriptions of links to other sites

– pattern fields (headers and trailers that appear on every page at the site)

Mining for data

• Test hypothesis• Look for hidden information• Find other similar information

Display of Information

• Graphical• Text Listing

– Directories: human maintained categories• e.g.: recreation, computers, finances, arts

– Computer generated list

• Customized– User defined defaults

– Cookie defined defaults

Data and Services

DisplayResultsLearning

to useservices

Learning to extract

data from the answer

Compileand clean

What Services?

• Search Engines• Internet White Pages

– (information on individuals)

• Internet Yellow Pages – (information on corporations)

• Usenet News repositories• Online libraries• Online periodicals

Learning to use Services

• Sample sets of data– can derive a format if taught to.

• Machine learning (same as in Data Mining)– look at every interpretation, find the one that conveys the

most information.

Learning to interpret answers

• What format is information given in?• What do the fields mean?

– Can identify unknown fields by matching the data with a known information.

Compile and Clean Data

• Redundancies• Duplicates• Redundancies• Newer information has precedence

Security

• Server environment– Use trusted CGI scripts and server side includes

• Client environment– Restrict access by IP number or domain

– Restrict access by password

• Internet– encrypt data (PGP)

– Certification authority

Data is in database?

Checking for hidden information

MachineLearning

Article: 52151 of comp.lang.perl.miscPath: lynx.unm.edu!pr1.plk.af.mil!tesuque.cs.sandia.gov!sloth.swcp.com!news.ironhorse.com!op.net!news.mathworks.com!enews.sgi.com!news.sgi.com!mr.net!news.mid.net!sbctri.tri.sbc.com!newspump.wustl.edu!newsfeed.rice.edu!rice!addFrom: add@pecos.is.rice.edu (Arthur Darren Dunham)Newsgroups: comp.lang.perl.misc,comp.infosystems.www.authoring.htmlSubject: Re: WWW: web site "pre-processor" in perl ?Date: 31 Oct 1996 00:20:06 GMTOrganization: Rice UniversityLines: 23Message-ID: <558rbm$61k@listserv.rice.edu>References: <539045$3de@news1-alterdial.uu.net> <8cvicmud6t.fsf@gadget.cscaper.com> <slrn55ih2s.qs8.charlie@antipope.demon.co.uk> <53o93k$3ia@panix.com>NNTP-Posting-Host: pecos.is.rice.eduXref: lynx.unm.edu comp.lang.perl.misc:52151 comp.infosystems.www.authoring.html:111886

In article <53o93k$3ia@panix.com>, Clay Shirky <clays@panix.com> wrote:>>Au contraire. HTML _is_ broken, relative to, say, SGML, but if you are>careful with your tags and comment carefully, your data can be derived>from your HTML files, not v-v.>>find . -name '*html' -exec perl -p -i.bak -e> 's#(<body[^>]*bgcolor="?)oatmeal("?[^>]*>)#$1skyblue$2#i;' {} \;

or if you wanted perl to do all the work, rather than have find(1)launch N perl executables for each .html files, you could do this....

find . -type f -name '*html' -print | xargs perl -p -i.bak -e 's#(<body[^>]*bgcolor="?)oatmeal("?[^>]*>)#$1skyblue$2#i;'

That way, perl happily iterates through all the lines in all the filessince we don't care which file we're in when we do the substitution.

-- Darren Dunham add@is.rice.eduUNIX Sysadmin Rice University(This line currently in revision) Houston, TXAny resemblance between real opinions and my post is coincidental

<HTML><HEAD><TITLE>Information gathering</TITLE></HEAD><BODY><TABLE><TR><TH><IMG SRC="info.gif"></TH> <TH><font size="+3">Information Gathering</font><BR>Just some sample text which might or might not be worthless.You'd want to sort out which of this was just HTML tags and other worthless junk and which was meaningful.</TH></TR></TABLE><P><CENTER><H2>Links to</H2><A HREF="/sameplace/otherinfo"> A link to something on this site </A>

<A HREF="/otherplace/otherinfo"> A link to something on this another site </A>

</BODY></HTML>

Re: Scots and English Gregory J Dalley, 30 May 1995, Lines: 18.Re: Dutch and English accents Phil Rose, 15 Jun 1995, Lines: 28.Re: ANY SIL'rs out there? A.K.A. Summer Institute of Linguistics. yomomma, 16 Jun 1995, Lines: 6.Re: ANY SIL'rs out there? A.K.A. Summer Institute of Linguistics. yomomma, 16 Jun 1995, Lines: 6.Conferences, Seminars-info wanted chris bowen, Mon, 03 Jul 1995, Lines: 7.AIGH? Coby (Jacob) Lubliner, 8 Jul 1995, Lines: 8."Shall" and "Will" in Welsh English maryproto@delphi.com, Wed, 19 Jul 95, Lines: 14.careers in linguistics scharle, 10 Sep 1995, Lines: 8.job opportunities in computational linguistics? Sonny Xuan Vu, 30 Sep 1995, Lines: 14.Re: job opportunities in computational linguistics? Miss Sarah Tiller, Wed, 4 Oct 1995, Lines: 27.Re: What Is Singapore English? Zhong Qiyao, 11 Dec 1995, Lines: 28.Re: What Is Singapore English? Chew Kim Swee Andrew, 14 Dec 1995, Lines: 41.Re: What Is Singapore English? Pota alok Ashwin, 16 Dec 1995, Lines: 45.Re: How to write in English ... Ann Weiner, Tue, 2 Jan 1996, Lines: 13.Re: What Is Singapore English? Wing Luk, 7 Jan 1996, Lines: 27.Linguistics Careers lebitz,stacey b, 23 Jan 1996, Lines: 14.English Teaching Offering in China - offer2.doc [1/1] XIAOJUN ZHANG, 24 Jan 1996, Lines: 240.TRYING TO PROTECT YOUR WORK? prepaid, Sun, 04 Feb 1996, Lines: 1.Give me, please, one program for learn to speak english!! Please!! "Eugen I. Ivanov", 20 Feb 1996, Lines: 1.Re: The English "R" for Germans Joerg Settemeyer, 8 Mar 1996, Lines: 5.English Tutor Needed. Mua Tran, 23 Mar 1996, Lines: 20.Re: old form of shorthand Fido, 1 Apr 1996, Lines: 9.Re: Math as pornography Gordon Fitch, 17 May 1996, Lines: 7.Re: Chain Shift Charles Lieberman, 26 Jul 1996, Lines: 10.Re: Tendency of Inflections to Disappear - Why? Terrence Griffin, 28 Jul 96, Lines: 1.Re: Concerning the number of esperantists Marc Bonnaud, Fri, 09 Aug 1996, Lines: 14.Re: Concerning the number of esperantists Cheradenine Zakalwe, Fri, 9 Aug 1996, Lines: 16.Re: Concerning the number of esperantists Alan Gould, Sat, 10 Aug 1996, Lines: 22.Re: Concerning the number of esperantists Don HARLOW, Sun, 11 Aug 1996, Lines: 21.Re: Kiom da E-istoj *ne* regas la anglan? Andrew McConnell, Fri, 30 Aug 1996, Lines: 19.cohesion in CMC Per-Mikael Jansson ENGE, 22 Oct 1996, Lines: 10.

Articles from sci.lang selected through webSOM

Limitations of the Web

• Some functionality/specialization was given up for ubiquity

• Transfer time– Mass data transfer prohibitive

• External to machine– Reliance on network

• Not inherently as secure as staying home

Why Data Mining

• There is a lot of data of unknown worth and purity• Data mining uses the same underlying procedures as

other knowledge discovery/ data extraction systems

Automatic Customization to user preferences

• Web pages– Hotwired autoconfigs based on what you surf to

• News services– usenet service custom.roy-corey.1

• Information display paradigm– industry report style

– collegiate style

– Microsoft style

Methods for gathering data

• Extraction from documents– data mining

– keyword searches

– similarity searches

• Extraction from services– ILA: internet learning agents

– Softbots

– Metacrawler

Data mining on the web?

• Transfer rate too slow to transfer most databases whenever you want

• Computation too intensive to let others mine your database whenever they want

• So: Use pre-collected data or pre-indexed database

Java -- What is it?

• Programming Language• Java Compiler• Java Interpreter (Java Virtual Machine)• For creating applets which run inside a browser• For creating applications (stand alone programs)

Java Application Source Code

// Sample HelloWorld application

class HelloWorldApp {

public static void main(String args[]) {

System.out.println("Hello World!");

Java Applet Source Code

// Sample HelloWorld applet

import java.awt.Graphics;

import java.applet.Applet;

public class HelloWorld extends Applet {

public void paint (Graphics g){

g.drawString("Hello world!", 25, 25);

How could you use it?

• Client applets or applications• Server code• Portable code• Create via Developer Tools

Developer Tools

• Visual C++ (Visual Java?)• Symantec• Sun• SGI - Cosmo Code

Developer Tools

• SourceCraft• Powersoft - Fusion• Quintessential Objects - Diva for Java (Javaside)• Roguewave - JFactory

Advantages

• Object Oriented and event-driven• Portable* bytecode• Multi-threaded• Integrated Network Abilities• Built-in Multimedia Capabilities• “Robust and Secure”

Drawbacks

• Few deployed clients• Very C++ -like• Not yet stabilized• Very few Developer Tools• Not all the class libraries exist (yet)

Class Structure

Class java.applet.Applet

java.lang.Object

+----java.awt.Component

+----java.awt.Container

+----java.awt.Panel

+----java.applet.Applet

Security

• OS security in applications• “No Pointers” and no user memory management• Compile-time and Run-time checking• Client Data Security

– No access to disk from Netscape

– Directory-based security in Hot Java

Security

• Network Security– No Applets

– No Access

– Applet Host

– Firewall

– Any Host

Security Problems

• CERT 96.05 - Firewall Security– ftp://info.cert.org/pub/cert_advisories/CA-

96.05.java_applet_security_mgr

• CERT 96.07 - Bytecode Verifier– ftp://info.cert.org/pub/cert_advisories/CA-

96.07.java_bytecode_verifier

Alternative Options

• Visual Basic and browsers• Visual Basic separate from WWW• Web Server without Java

Books About Java

• Teach Yourself Java in 21 Days• Java!• Hooked On Java• Presenting Java• O’Reilly

Java WWW Sites

• Sun– http://java.sun.com/

• The Internet Programming Page– http://www.apexsc.com/vb/internet.html

• Rogue Wave Home Page– http://www.roguewave.com/

• Symantec Café– http://cafe.symantec.com/cafe/index.html

Java WWW Sites

• JavaSoft– http://www.javasoft.com/

• The Java Directory (Gamelan)– http://www.gamelan.com/

• IBM: Centre for Java Technology– http://www.hursley.ibm.com/javainfo/

• News: comp.lang.java

Internet and WWW Services

Documents

Internet & WWW: infraestructura del comercio

Internet and WWW Technologies

Internet services & WWW

Chapter1 - The Internet and WWW

W 11 internet & www

Internet, WWW Retele comunicationale

Broadcast Technology RTV 151 Telephony Internet & WWW Broadband etc. Telephony Internet & WWW Broadband etc

Internet / Protocols / WWW

Technology Hardware Internet & WWW. Outline Hardware System Unit CPU Memory Ports Internet Internet services WWW Types of Sites E-Commerce

CONTENTS Definition And History Basic services of INTERNET The World Wide Web (W.W.W.) WWW browsers INTERNET search engines Uses of INTERNET

Citw Internet & Www

Internet dan www

Eisagogi sto internet-email-www

MELJUN CORTES internet and www

ALFIN 1 - Internet y WWW

Internet e Www

Chapter 02 - Internet & WWW

MELJUN CORTES Internet & WWW

Internet (www)[1]

1/28: The Internet & Website Design What is the Internet? –Parts of the Internet –Internet & WWW basics –Searching the WWW Website design considerations