All Case Studies Design Development Interviews Our Way Project Management

Talks and Startups That Rocked (my) WebSummit

It’s hard to summarize the biggest technology conference in Europe. Every topic covered and 100’s of amazing tech business from every sector (including Inn Style where I had the pleasure of working as a junior developer).

It’s hard to summarize the biggest technology conference in Europe. Every topic covered and 100’s of amazing tech business from every sector (including Inn Style where I had the pleasure of working as a junior developer).

What is evident is data is everywhere. There’s so much data it’s hard to know what to do with it all. A lot of interesting businesses are coming up with some novel analytics solutions based on automation.

And Don O’Neill from Trip Advisor was talking about the topic. Trip Advisor save a lot of data. They get 1.5 million hits per minute on over 1000 servers which generates more than 2 terabytes of compressed logs per day for them!

Their motto is:

“If it’s worth doing, it’s worth measuring.”

And according to Don, it’s not about hiring experts in big data or statistical modeling. It’s about finding things that can be repeated and automating them. And they are proving it’s possible: Trip Advisor have just 12 of their regular software engineers who are also working on Dev Ops!

I liked Don’s honesty in the talk. He shared with us some of the things he learned from some of the challenges they faced. He said their deployment process had been less than optimal... The deployment process was very frustrating for the developers.

They had 3 deployments per week and pushing the code to production went as follows.. push the code and pray! The team would spend 3 hours watching the servers checking some metrics in Cacti and looking for new errors because their integration testing wasn’t testing the deployment at scale. Cacti is a tool that monitors basic server metrics (around 30) and over a pool of 1000’s of servers there must be a better solution than just comanalysing these metrics.

The team came up with a new process that reduced their watching the production server from 3 hours to 15 minutes. This time the code deployed to just 2 / 1000 pools of servers. Comparing the metrics between the new and old deployments with automated analytics could then be easily checked for differences resulting in red or green. If something went wrong they automatically rolled back the deploy to the earlier version.

The new process quickly moved away from Cacti. They used Collectd to gather all the real time metrics. They sent logs to Graphite for analytics. What they ended up with was an awesome visualization of server performance over time which could easily show the differences in performance for a new deployment compared with the current one and automatically roll back any poor performing deployments.

They also gathered analytics data that could show differences in request response times between the newly deployed code which made it easy to see performance issues.

They also used Postgres for anomaly detection on log data. Postgres is a powerful engine for running calculations.

Benefits of AB Testing

Dan Siroker (Optimizely) was also showing the value of data analytics. He explained how important AB testing was during the Obama campaign. Through AB testing the SignUp button and the home screen image they increased user signups by an enormous amount. The catalyst for using AB testing was when he asked the audience to choose what they thought was the winning SignUp text and Photo, and only one person got it right, a clear win for AB testing.

And more visualizations/analytics

Trifle.io, a Ruby on Rails startup is developing a new analytics solution for Rails apps. More than the basic New Relic analytics, this app aims to show exact issues in method calls, database calls and will have a much more affordable pricing model.

Christoph Holz from visalyze.com showed some impressive data visualization. He should get in touch with Trifle because the two could make a good combination.

s3

Photo by William Murphy.

BLRT is a new app that allows for amazingly low data bandwidth for audio visual communication (which isn’t in real time). Looks great for talking over mockups and websites using arrows and voice.

Pushstartr.com have a solution for easily dealing with multiple database technologies. Sounds very useful if you need to transfer your old mysql database to use Postgres on Heroku or even Mongo.

If you've enjoyed this sum-up, make sure to read another inspiring story form Dublin Web Summit, this time about Leland Melvin - NFL star and NASA astronaut

Why it is impossible to hire quality engineers in London and how to deal with it
Follow Netguru
Join our Newsletter

Scaling SaaS
READ ALSO FROM Events
Read also
Need a successful project?
Estimate project or contact us