11
BigData Solutions HIGH TECH GENERATION SOFTWARE No 6. Phan Dinh Giot, Tan Binh, HCMC Email: [email protected] Phone: +84 909 087056 Date: Apr 02, 2015 Version: 1.0 Last Update: Apr 02, 2015 Prepared for: New Customer Prepared by: H H. Le (Product Manager) TABLE OF CONTENTS 1. Project Overview ............................................................................................................... 2 1.1. Business Cases ..................................................................................................................................... 2 1.2. Platform & Devices ............................................................................................................................ 2 2. Technical Requirement ................................................................................................... 3 2.1. Goals of Architecture ........................................................................................................................ 3 2.2. Endusers Requirements ................................................................................................................ 5 3. Technical Approach .......................................................................................................... 6 3.1. Overview Architecture ..................................................................................................................... 6 3.2. Server Side ............................................................................................................................................ 6 3.3. Client Side .............................................................................................................................................. 9 4. Services information ..................................................................................................... 10 4.1. Packages ............................................................................................................................................... 10 4.2. Development Process ..................................................................................................................... 10 4.3. Human Resource .............................................................................................................................. 10 4.4. Timeline................................................................................................................................................ 11 4.5. Budget ................................................................................................................................................... 11 4.6. Payment Method..................................................................Error! Bookmark not defined.

Big data solutions - Big data technology

Embed Size (px)

Citation preview

Page 1: Big data solutions - Big data technology

 

 BigData  Solutions  

 

HIGH  TECH  GENERATION  SOFTWARE  

No  6.  Phan  Dinh  Giot,  Tan  Binh,  HCMC  

Email:  [email protected]  

Phone:  +84  909  087056  

Date:  Apr  02,  2015  

Version:  1.0  

Last  Update:  Apr  02,  2015  

 

 

Prepared  for:     New  Customer  Prepared  by:     H  H.  Le  (Product  Manager)  

 

TABLE  OF  CONTENTS  

1  . Project  Overview  ...............................................................................................................  2

1.1. Business  Cases  .....................................................................................................................................  2

1.2. Platform  &  Devices  ............................................................................................................................  2

2  . Technical  Requirement  ...................................................................................................  3

2.1. Goals  of  Architecture  ........................................................................................................................  3

2.2. End-­‐users  Requirements  ................................................................................................................  5

3  . Technical  Approach  ..........................................................................................................  6

3.1. Overview  Architecture  .....................................................................................................................  6

3.2. Server  Side  ............................................................................................................................................  6

3.3. Client  Side  ..............................................................................................................................................  9

4  . Services  information  .....................................................................................................  10

4.1. Packages  ...............................................................................................................................................  10

4.2. Development  Process  .....................................................................................................................  10

4.3. Human  Resource  ..............................................................................................................................  10

4.4. Timeline  ................................................................................................................................................  11

4.5. Budget  ...................................................................................................................................................  11

4.6. Payment  Method  ..................................................................  Error!  Bookmark  not  defined.

 

Page 2: Big data solutions - Big data technology

 

 2  

1  . Project  Overview  

1.1. Business  Cases  

For  Developer  

- Access  world  wide  agents  

- Stock/Invetory  management  

- Track/Log  user  viewed  project  

- Reporting  tools  &  insights  

- Sale  &  Marketing  management  

- Instant  email  notifications  

Distribution  Network  

- Search  projects  

- View  live  price  lists  

- Access  project  information  

- Reserve  direct  with  the  seller  

- Sale  &  Marketing  management  

- Contract  management  

1.2. Platform  &  Devices  

Web  Responsive.  

- 2  versions  (PC  +  tablet  and  mobile)  

- Display   compatibility:   Internet  Explorer  9.0+,   Firefox  15+,   Chrome  20+,   Safari  

6+  

Mobile  Application:  

- IOS  version  6  &  above  

- Androidversion  4.1  &  above,  iOS  

   

Page 3: Big data solutions - Big data technology

 

 3  

2  . Technical  Requirement  

2.1. Goals  of  Architecture  

The  architecture  has  been  designed  to  build  a  more  reliable,  robust  and  scalable  

distributed  systems.  Below  are  our  goals  and  principles  of  the  cloud  base  architecture:  

 High  Availability:  

An   global   marketplace   for   off-­‐the-­‐plan   properties   must   be   a   non-­‐stop   service.  

Because   of   being   unavailable   for   even  minutes   can   result   in   thousands   or  millions   of  

dollars   in   lost   revenue.   So   that   the   uptime   of   the  website   is   absolutely   critical   to   the  

reputation  and  functionality  of  the  website.  We  can  say  the  system  must  be  available  for  

24  hours  a  day,  7  days  a  week  and  365  days  a  year.  

Page 4: Big data solutions - Big data technology

 

 4  

 Robust  Performance:  

The   speed   of   the  website   affects   usage   and   user   satisfaction,   as  well   as   search  

engine  rankings,  a   factor   that  directly  correlates   to  revenue  and  retention.  As  a  result,  

the   architecture   of   the   system   is   optimized   for   fast   responses   and   low   latency.   The  

threshold  of  response  time  should  be  <  5  seconds  (5s)  per  request.  

More  Reliability:  

A  system  needs  to  be  reliable,  such  that  a  request  for  data  will  consistently  return  

the  same  data.  The  reliability  of  the  system  requires  the  data  must  be  consistent  and  the  

high  read  and  write  performance  under  highly  concurrent  conditions.  User’s  transaction  

like  reserving  an  apartment  or  update  commission  rate  must  be  correctly  handle.  Every  

searching  requests  must  return  the  accuracy  results.  

Fast  Scalability:  

The   effort   required   to   increase   capacity   to   handle   greater   amounts   of   load,  

commonly   referred   to   as   the   scalability   of   the   system.   Scalability   can   refer   to   many  

different  parameters  of  the  system:  how  much  additional  traffic  can  it  handle,  how  easy  

is   it   to   add   more   storage   capacity,   or   even   how   many   more   transactions   can   be  

processed.  

Ease  Manageability:  

The   manageability   of   the   system   equates   to   the   scalability   of   operations:  

maintenance   and   updates.   The   goals   are   the   ease   of   diagnosing   and   understanding  

problems  when  they  occur,  ease  of  making  updates  or  modifications,  and  how  simple  the  

system  is  to  operate.  

Page 5: Big data solutions - Big data technology

 

 5  

Predictable  Cost:  

The  system  should  be  guaranteed  for  predictable  costs  with  no  surprises  or  over-­‐

use  costs.  In  additional,  the  system  leverages  the  cloud  computing  and  it  will  be  different  

with  traditional  system.  The  cost  of  traditional  system  will  be  fixed  cost  because  owner  

must  buy  the  servers  and  hire  a  team  to  operate  the  systems.  The  predictable  cost  cloud  

system  must  be  a  variable  cost  and  the  cost  will  be  only  increase  with  the  increment  of  

the  revenue.  

2.2. End-­‐users  Requirements  

Beside   the   technical   requirements,   the   architecture   should   be   designed   to  

support  the  end  user.  There  are  list  of  end-­‐users  requirements:  

Internationalization  And  Localization  (i18n)  

The  system  serves  world  wide  users  who  come  from  many  countries  and  speech  

many  languages.  To  help  user  to  be  combatable  and  easy  to  use  the  system,  the  website  

must  have  ability  to  translate  to  local  languages;  support  local  time-­‐zones  (e.g.:  GMT+8  

for  China  or  Daylight  Saving  Time  for  Australia,  …);  support  multiple  number/currency  

formats,  date  time  formats;  ability  to  display  text  from  right-­‐to-­‐left  for  Dubai  customer  

may  be.    

Responsive  and  mobility  display:  

The   system   must   well   display   on   a   rich   of   devices   as   smartphones,   smart-­‐

watches,  tablets,  big/high  resolution  screens  or  projectors.    

Security:  

The  system  must  have  a  strong  and  flexible  access  control  list.  Each  of  developer  

or  property  seller  have  a   fully  controlling  and  granting  access   to   their   sale  properties.  

Other   roles   of   agencies,   accountants,   advisors   or   financial   planner  will   have   a   secure  

access  to  the  system.    

Page 6: Big data solutions - Big data technology

 

 6  

3  . Technical  Approach  

3.1. Overview  Architecture  

 MVC  Model.  

Cloud-­‐oriented,  fully  support  with  Amazon  Web  Service.  

Capability  to  widen  application,  and  integrate  with  other  system.  

3.2. Server  Side  

Using   PhalconPHPhttp://phalconphp.com/en/   framework   (latest),   MVC   model,  

fast,   robust   and   reliable   web   framework,   easy   to   understand   and   apply,   completely  

support   RESTful   (easy   to   develop   mobile   application   or   allow   other   systems   to  

integrate).  

Apply  Role  Based  Access  Control  design  pattern  to  secure  the  whole  application.  

Some   complicated   operations   such   as   concurrency   update,   transactional  

management  will  be  handled  with  Phalcon  ORM  framework.    

OS  /  Hosting  

We  run  Ubuntu  Linux  14.04  (“Trusty  Tahr”)  on  Amazon  EC2.  

Load  Balancing  

Page 7: Big data solutions - Big data technology

 

 7  

 Using  Amazon’s  Elastic  Load  Balancer,  with  1-­‐3  Apache2  instances  behind  it  that  

can   be   swapped   in   and   out   (and   are   automatically   taken   out   of   rotation   if   they   fail   a  

health  check).  

Application  Servers  

We   run   PhalconPHPhttp://phalconphp.com/en/   on   Amazon   Medium-­‐CPU  

machines,  and  as  our  usage  grows  we’ve  gone  from  just  a  few  of  these  machines  to  over  

10  of  them.  It  requires  the  designing  system  should  be  easy  to  horizontally  scale  as  they  

are  stateless.  

Big  Data  Processing  System  

We   use   the   graph   database   to   build   an   intelligent   system.   Some   features   like  

recommendation   with   ability   to   recommend   suitable   properties   for   an   investor   or  

recommend  a  lead  customer  to  an  agencies.  

Most   of   our  data   (users,   photo,   tags,   etc)   lives   in  MariaDBhttps://mariadb.org/  

that   run   on   Amazon  High-­‐CPU   Extra-­‐Large   EC2   instance  with   EBS   optimized   for   high  

input.    

 We  also  use  Redishttp://redis.io/  to  powers  our  session  system  and  task  queue.  

Page 8: Big data solutions - Big data technology

 

 8  

For   our   geo-­‐search   API   and   real   estate   properties   full-­‐text   search   we   use  

ElasticSearch.  

We  use  Memcachedhttp://memcached.org/  for  caching  data,  html,  etc.  

Amazon  S3  will  be  used  for  storing  properties  images.    

Task  Queue    

 We   use   Resquehttp://resquework.org/   for   handling   background   jobs.   We   also  

have  Resque  Dashboard  for  monitoring.  Our  cloud  base  scalable  and  robust  task  queue  

system  will  help  the  system  to  handle  unlimited  user’s  requests  at  any  time.  

Logs  and  Monitor    

 The   system   logs   and   application   logs   will   be   written   to   RSyslog.   Logstash  will  

Page 9: Big data solutions - Big data technology

 

 9  

collect   logs   from  RSyslog  and   crunch   them   to  ElasticSearch.  Kibana  dashboard  will   be  

used  to  monitor  the  whole  system.  

3.3. Client  Side  

Using  jQuery  and  CanJS  support  HTML5  and  CSS3.    

Twitter  Bootstrap  3  latest  will  be  used  for  responsive  web  design  

Page 10: Big data solutions - Big data technology

 

 10  

4  . Services  information  

4.1. Packages  

- Backend,  APIs  

- Frontend  

- Android  App  

- iOS  App  

- User  Experience  

- Design  

4.2. Development  Process  

HTGSoft  adopts  Agile  methodology  and  relies  on  Scrum  in  the  software  delivery.  

Agile  Scrum  requires  the  clients  to  be  involved  in  the  whole  project  development  cycle  

from  the  very  outset.  It  allows  the  team  to  collect  customer  feedbacks  quickly  then  dealt  

with  changes  flexibly  and  instantly  to  make  sure  that  the  released  products  will  be  fitted  

to  user  expectations.  

With   Agile   Scrum,   HTGSoft   can   achieve   highest   productivity   in   complicated  

projects.  

   4.3. Human  Resource  

#   Name   Title   Role  

1   A   Product  Manager   Scrum  Master,  Project  Manager  

2   B   Technical  Manager   Technical  Lead  

Page 11: Big data solutions - Big data technology

 

 11  

3   C   Technical  Expert   Developer,  Product  Owner  

4   D   Senior  Engineer   Developer  

5   E   Senior  Engineer   Developer  

6   F   QC  Engineer   Quality  Consultant  

7   G   QC  Engineer   Quality  Consultant  

8   H   Technical  Expert   System,  Security  

9   I   Engineer   Developer,  System  

10   J   Engineer   Developer,  System  

11   K   Senior  Engineer   Android  

12   L   Engineer   Android  

13   O   Senior  Engineer   iOS  

14   P   Engineer   iOS  

15   Q   Business  Analyst   UX  Specialist  

16   R   Senior  Designer   Designer  

17   S   Senior  Designer   Designer  

 4.4. Timeline  

Attached  Document:  xxxProject  Timeline.pdf  

4.5. Budget  

Attached  Document:  xxx_option1.pdf