MarkLogic 8: Samplestack...With MarkLogic 8 we’re moving the product and how we talk about it to where mainstream organizations already are. These are th\ 99.9999% of organizations
MarkLogic 8 Developer Experience Objectives Better align with how software is built, tested, integrated, deployed, and
maintained
Lower barriers to entry: Success without supervision
Make it easier to attract and retain great developers
More reusable patterns and tools
No compromises on performance, reliability, differentiators
Presenter
Presentation Notes
A new focus on three-tier architecture is just part of the story of making MarkLogic easier to understand and want to use—to love. As with all of our big product investments, the things we’re doing around making MarkLogic more attractive to developers and easier to adopt will span multiple releases. We often hear people use “ease of use” as short-hand but “ease of use” undersells MarkLogic. We don’t want to be gratuitously difficult to use. We want to focus on making developers want to build their next application with MarkLogic. That could be because it was easy to use, but also because it will look good on a resume or buy political capital with the ops team or mean writing orders of magnitudes less code to achieve the things an app needs to do.
New and Enhanced Features in MarkLogic 8 Samplestack: Reference architecture instance
Client APIs: Extensible drivers for Java and Node.js
Open development: Transparency and responsiveness with GitHub-first
Server-Side JavaScript: Not XQuery
Native JSON: Unified indexing, query for today’s data
Presenter
Presentation Notes
So, how does that shake out it what we’re doing with the product? In the next two hours or so we’ll work our way down the stack, the same way that we’ll be presenting these concepts to developers, architects, and admins.
With MarkLogic 8 we’re moving the product and how we talk about it to where mainstream organizations already are. These are the 99.9999% of organizations who aren’t yet our customers. This starts with a focus on three-tiered architecture for building applications. In our documentation, training, product offering, and marketing we’ll introduce MarkLogic to new users using the concepts and patterns that they already know. The point here is to not to dumb down MarkLogic, but to more quickly ease new users into our worldview with knowledge and skills they’re familiar with. This doesn’t mean that other architectures are no longer valid. It just means we’ll concentrate on a smaller set of well known choices for new customers and projects as a starting point. We’ll build up samples, demos, training, documentation, sales tools, etc. around this, reducing confusion for new users and allowing us to more highly leverage marketing, sales, and engineering resources.
One of the drivers behind the tier architecture, and one of the reasons for its staying power, is to factor out the main functions of an application stack into components that can developed, managed, and scaled independently. At the bottom there’s the database. It maintains shared state throughout the stack. In a database like MarkLogic, it can also run specialized logic close to the data, encapsulating functionality and saving a serialization and network hops. Above the database is application middleware. This middle tier is responsible for handling state changes from the front-end and enforcing business rules, for example, not selling more widgets than you have on-hand in inventory. Middleware is the friendly face to the persistent data, providing services in the language of the business, enforcing rules, and orchestrating interactions with other systems. Finally at the top of the stack is the UI. . The UI is where users interact with data. Increasingly the workflow of how a user progresses through an application is managed in the UI. The database and the middle tier typically communicate in what I call “data services”. These are things like documents and collections in MarkLogic or ResultSets and columns in the relational world. In MarkLogic these services would typcially be exposed as XML or JSON over HTTP. The middle tier, as broker between the UI and database, is responsible for distilling data services into, what we’re calling “business services”. These are nouns and verbs in the language of the business domain, things like orders, customers, users, entitlements. Given the predominance of JavaScript in the browser, these business services are typically exposed as JSON over HTTP. Of course, the lines between each of these tiers may be blurry. A stored procedure in the database may encapsulate logic that would normally be maintained in the middle tier. The rich capabilities of web and mobile clients have pushed much of the workflow and domain model from the middle tier to the user interface. As a database, MarkLogic fits well into a traditional three-tiered architecture. At the top of the stack is the user interface. The UI is where users interact with data. Increasingly the workflow of how a user progresses through an application is managed in the UI. The UI typically communicates with middleware over what, I call "business services". The predominance of web development has resulted in most of these services being JSON over HTTP. At the other end, middleware is responsible for handling state changes from the front-end and enforcing business rules, for example, not selling more widgets than we have on-hand in inventory. Middleware is the friendly face to the persistent data, providing services in the language of the business, enforcing rules, and orchestrating interactions with other systems. The database provides persistence storing shared state for the layers above it. By taking care of things like transactions, security, and distributing workloads efficiently among available resources, MarkLogic unburdens developers from having to manage these complex concepts in application code (or not at all). Who out there has written a distributed transaction controller or compartment-level security? Building these in middleware is risky and expensive business and probably doesn't match the skillsets of your developers.�The database communicates with the middle tier in "data services". While the middle tier and UI communicate in terms of domain objects, like "Customers" and "Approvals" or "Articles" and "Entitlements", the middle tier and the database communicate in terms of documents and collections or elements and properties.
This architecture applies equally well if you swap out a service endpoint for a graphical user interface. Link multiple of these together with some orchestration and messaging and you’ve got a service-oriented architecture.
SAMPLESTACK
Presenter
Presentation Notes
Samplestack is an accelerator, hands-on, it’s an application but its meant for developers…but what do you GET when you use it? What are the benefits?
An end-to-end three-tiered application in Java and Node.js
Front End
Middle Tier
Database Layer
Encapsulates best practices and introduces key MarkLogic concepts
Use sample code as a model for building applications more quickly
Modern technology stack shows where MarkLogic fits in your environment
Participate. Contribute. Fork it.
Samplestack
Presenter
Presentation Notes
Get going fast on MarkLogic with Samplestack, an end-to-end three tiered sample application designed to show developers how to implement a reference architecture using key MarkLogic concepts and sample code. The Reference Architecture provides a blueprint for building three tiered applications on MarkLogic. Samplestack takes the reference architecture from theory to practice. Samplestack is a fully featured execution of that architecture that demonstrates with working code how applications run on MarkLogic. New to NoSQL? Or document stores? Samplestack will introduce you to the most important concepts of MarkLogic and present tangible development patterns to follow. Samplestack is an accelerator that gives Developers the ability to become proficient with MarkLogic quickly. See how the pieces of the architecture fit together in your own environment, using familiar tools and modern technologies. Samplestack features an Angular.js front end with two versions of the middle tier in the MarkLogic 8 release: There is a Java/Spring version with Gradle automation [which we’ll cover today] And a JavaScript/Node.js version with Gulp automation [in progress and coming soon]. Both versions of Samplestack leverage the same MarkLogic database configuration The source code as well as on-going development efforts can be found on GitHub as this is a community-facing project. Iterative so we are responsive and continuing to post updates on our own timeline. Developers can instantly dive in and explore the Samplestack source code, no training or reading documentation required. Jump start your project by using the features introduced in Samplestack as a starting point for your own MarkLogic apps.
Samplestack unlocks MarkLogic’s power Two paths to exploring MarkLogic – via the fluent Client APIs in Java and Node.js
Emphasizes the role of the middle tier (for authentication and business services) and highlights interactions with the database and front-end
Learn how to make data modeling easier using relationships and native document formats (JSON, POJOs, and RDF)
See how MarkLogic’s built-in search and indexing capabilities work
Get an introduction to Semantics by seeing how Samplestack links related documents with SPARQL queries
Presenter
Presentation Notes
With Samplestack, we’ve distilled the power of MarkLogic into one easy to grasp application concept. By exploring the code you’ll learn valuable concepts that are portable to other applications you might build Learn new things about MarkLogic by starting from a familiar point of exploration with the Java and Node.js client APIs Study the architecture and notice the similarities to what is typical in the marketplace, also nuances of our model and architectural decisions. See why the NoSQL document model is better than a purely relational one by looking at how Samplestack models documents in JSON, uses objects, and models relationships separately in RDF Look at how Samplestack uses built-in capabilities for search, indexing and facets in order to more quickly deploy your own search or content application Soon, you will even be able to get a feel for how MarkLogic Semantics works in the application by checking out the SPARQL queries and use of RDF triples. This is one of our newer features that is still in progress.
Sample application with enterprise-grade features Transactional multi-document commits
Demonstrates the security model with role-based privileges and document permissions
Supports agile development throughout the cycle from build automation through test
MVC framework separates concerns and eases maintainability and extensibility
Presenter
Presentation Notes
Although Samplestack was developed as a teaching tool and is a sample application, architectural decisions were made to enable future scaling of the app for production environments. It shows off MarkLogic’s robust enterprise-hardened platform and is a great foundation for developers. Learn how to build applications the right way from the start. Just because it’s a sample application developed on a NoSQL platform doesn’t mean sacrificing ACID transactions. CRUD Operations are transactional whether they impact a single document or span multiple and Samplestack teaches you how to use that in your application. Samplestack employs simple yet robust security and exposes how MarkLogic’s underlying role-based model functions in a working application. We’ve even shown how to integrate with LDAP. In addition to the flexible, schema-agnostic data model, Samplestack incorporates other features to show how application development with MarkLogic can fit into your development. Build automation using common tools like gulp and gradle show how the REST Management API supports scripted deployment. Unit and end to end tests have been included to show best practice development. Learn how an MVC application fits naturally on MarkLogic with a real-life example. It demonstrates a flexible architecture that can be modified for different configurations, browsers, and build components.
Samplestack features an Angular.js front end and two versions of the middle tier: Java/Spring with Gradle automation JavaScript/Node.js with Gulp automation The technologies are meant to be forward-looking and modern. Made intentional decisions and have largely heard positive feedback from customers using or considering similar architectures. Shouldn’t be seen as a constraint as the concepts are portable – can use the same set up on the REST API, or with a different browser technology. --- Finally, we're pulling together all of the concepts we've discussed today into a reference architecture that developers can install in their own environment to actually be able to touch and feel these architectural concepts in a real application. For those of us old enough to remember, Sun released a "Pet Store" application when they were first promoting J2EE. The application was trivial in its functionality, but demonstrated many of the key concepts that went into designing a real production application. We're taking these same objectives and updating them to a modern data-centered application with MarkLogic. With the reference architecture you'll get a three tiered application in both Java and Node.js. A developer will be able to download and deploy it in one or two command line calls. It will illustrate best practices for using our client API in middleware and demonstrate some of the core capabilities the show up in many MarkLogic applications, like faceted search, and bulk ingestion. It will also show best practices for designing unit tests at each tier and integration tests across tiers. We'll also include deployment and configuration automation to allow you to script. We'll base these off of the current state of the art in the Java and Node ecosystems, allowing you to plug MarkLogic administration into an existing workflow, beyond a sample application. We'll ship this from GitHub under an open-source license so you can fork and change the code as you see fit.
How will you use this? Sample code for accelerating learning how to develop applications on
MarkLogic
Foundation for building reusable tools and establishing patterns
See new MarkLogic 8 features in action:
– Enhancements to the Java API (POJO façade, bulk load)
– REST Management API (for gradle database configuration)
– Server-Side JavaScript
– Native JSON
Presenter
Presentation Notes
ML has great documentation, and there are already many resources for developers including our mailing list, the Stack Overflow community, tools, github open source projects. So where does this fit? Status Quo (pre-Samplestack) – own forks Be able to point to code that demonstrates concepts you talked about or whiteboarded with a customer/prospect. Tell the story of the feature: how it fits into the overall MarkLogic positioning, how we've differentiated our implementation relative to what else is out there, and use cases where this might be employed, all in concepts and language that an SE or consultant would be able to convey to an architect, developer, CIO, or other technical decision-maker.
Accepted Answers & Reputation → Transactional model, data integrity
Related tags → Semantics
User records and Q&A documents → Data Model for JSON and POJOs
Presenter
Presentation Notes
Application Feature table on EA website. Very tangible list of some of the most important concepts that can be gleaned from looking at Samplestack. Link to GitHub.
Samplestack is a Q&A Application. Based on Stack Overflow – data from there. Users can post questions, answers, vote up content. Search. Earn reputation. Tag content. Every feature has a purpose….kept it simple but broad enough to demonstrate the most important features that are common to MarkLogic applications. SEARCH – full text, syntax/search tips Facets – boolean, date histogram, tag filters. Search results – snippets, metadata. Sorting Ask a Question (not yet enabled) View a Doc Security – log in and show # of documents, privileges. Updates impact relevance – vote switch around, immediate results Transaction – reputation accepted answer Now a full-stack demo in 3 parts. Anatomy of a search in Samplestack. (“jquery" search text, javascript tag facet, date histogram dragged at beginning of 2012, sort by newest). What’s still to come? Node, Semantics, More tags, Date
Presenter
Presentation Notes
Q&A What’s changed Architecture/approach Resources – specs, links Short answers Gauge audience familiarity with Samplestack, interest in build process Code/spec walk-through / concept outline FAQs Canonical example, list of other topics could cover
