Nati Shalom (Gigaspaces) pulls out a couple of references from James Hamilton’s posts[1] on Amazon SimpleDB and Google Megastore consistency model concluding:
It is interesting to see that the reality is that even Google and Amazon - which I would consider the extreme cases for big data - realized the limitation behind eventual consistency and came up with models that can deal with scaling without forcing a compromise on consistency as I also noted in one of my recent NoCAP series
But he lefts out small details like these:
Update rates within a entity group are seriously limited by:
- When there is log contention, one wins and the rest fail and must be retried
- Paxos only accepts a very limited update rate (order 10^2 updates per second)
and
Cross entity group updates are supported by:
- two-phase commit with the fragility that it brings
- queueing ans asynchronously applying the changes
Original title and link: Amazon SimpleDB, Google Megastore & CAP (NoSQL databases © myNoSQL)
MongoDB at Shutterfly
We looked at just about everything we could think of. Lots of options like Cassandra made sense for a very narrow problem set, where MongoDB allowed for flexibility. We wanted that flexibility.
Video and slides, highlights, and an interview with the architect of Shutterfly, Inc.
Original title and link: MongoDB at Shutterfly (NoSQL databases © myNoSQL)
The Power of Why
I run a group in Portland called PDX Web & Design, where we recently went the lazy route and just did two hours of lightning talks. As the night wound down - and the drinking wound up - I decided to jump on stage and do a half-drunken impromptu talk about what I remembered from a year old Ted talk.
Simon Sinek, author of “Start With Why”, gave a presentation (which you should watch) last year tying great businesses and leaders to a fundamental plane. He calls this “The Golden Circle”, and has become so ingrained into my concept of business, leadership, and communication that it rarely occurs to me to discuss it with anyone. It underlies Inviite. Here it is:
I recommend everyone watch the talk, but the basic concept is that the gut-check portion of our brains - the “limbic brain” - is primarily concerned with “Why” (beliefs) answers. The “neocortex” portion of the brain is the intellectual and speech portion concerned with ”How” (tactics) then “What” (results) answers. What’s interesting is that, since “What things are” and “How things work” questions are connected to the language portion of the brain, it is more difficult to correctly answer “why” questions.
It’s no wonder the most difficult questions are generally “why” - such as ”Why are we here?” or “Why do bad things happen to good people”. Answers to those questions are often deeply cherished beliefs. “Why” answers generally are. When people agree with your “why”, they believe what you believe - and a loyal partner in a movement is much better than a customer.
Because of this difficulty, we often just skip the “why”, and go straight to the ”what” or ”how”. Consider this contrived example:
- “What do you do?” - “My company builds houses.”
- “How do you do that?” - “We prefabricate materials so it’s inexpensive.”
- “Why do you do that?” - “Um… I don’t know. To make money?”
But that doesn’t necessarily sell people on an idea - much less garner fans. Everyone has been in a situation where they attempt to convince someone with all of the logical reasons - only to be shot down by: “I understand your points, but it just doesn’t feel right.” That’s the limbic system at work - you haven’t given they a “why”. Consider the above exchange that starts with a “why”:
- “Why does your company exist?” - “We believe it’s everyone’s right to affordable shelter.”
Much more powerful. The “how” and the “what” logically follow. I can get behind that kind of a business, because it’s not just a company - it’s a movement.
Martin Luther King, Jr was by no means the only great orator of his day, nor was he the only civil rights leader. But he framed himself in a “why”. He believed the calling of his generation was that laws of a higher authority existed, and that man’s authority must match. His famous speech was “I have a dream”, not “I have a plan” - a “how” phrase which would inspire few people. Having an answer to “why” your business or group or leadership exists is a fundamentally stronger case than “here is how we will do it”, or “here is what we provide.”
As an example, here is a breakdown of why Inviite exists:
- Why? We believe gathering is an important and world-changing human right.
- How? We promote gathering by leveraging technology, thus making it easier to do meet.
- What? Inviite is an online tool which makes it easy to create and share events.
That’s why I do this. That’s also why I don’t care about your meetups (which I use), evites, or foursquares. They have the market - but I don’t understand why they do what they do… and I have a feeling neither do they. Inviite has a focus - and it’s not just a collection of random “me too” features or cloned ideas.
Options of running RDBMSs in the cloud:
- Install and Manage – in this “traditional” model the developer or sysadmin selects their DBMS, creates instances in their cloud, installs it, and is then responsible for all administration tasks (backups, clustering, snapshots, tuning, and recovering from a disaster. […]
- Use a Cloud-Managed DBaaS Instance – in this model the cloud provider offers a DBMS service that developers just use. All physical administration tasks (backup, recovery, log management, etc.) are performed by the cloud provider and the developer just needs to worry about structural tuning issues (indices, tables, query optimization, etc). […]
- Use an External Cloud-Agnostic DBaaS Solution – this is very much like the cloud-based DBaaS, but has a value of cloud-independence – at least in theory. In the long run you might expect to be able to use an independent DBaaS to provide multi-cloud availability and continuous operations in the event of a cloud failure.
I guess these are equivalent to applying Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Database-as-a-Service (DaaS) (nb: this can be seen as a more specialized PaaS) models to persistency. And the same approach applies to NoSQL databases, as these models are orthogonal the the persistency problem.
cloudbzz.com
Original title and link: SQL and NoSQL In the Cloud (NoSQL databases © myNoSQL)
Rails 3/S3/CloudFront Asset Management
Inviite uses SASS/Compass/Blueprint to generate CSS, CoffeeScript for Javascript, Jammit to package and compress separate files, and Amazon CloudFront CDN to host the files.
Looking for some methods to automate S3 deployment, I ran across this post to package assets For rails 3 and upload to Amazon s3. I tweaked it to use Rake::FileList, set gzip headers, and added tasks to kick off, CoffeeScript and Compass compilation, then run Jammit, which populates the public/assets directory with compressed (and gzipped) javascript and css files.
All of these assets are packaged and uploaded to S3 via:
rake cache:s3
Then comes CloudFront. Linking up an S3 bucket to point to CloudFront is relatively straightforward, and uploading files to S3 is how you populate the CDN. However, CloudFront only checks for file changes once every 24 hours, so I added a rake task to invalidate the CloudFront cache and force it to take the S3 uploaded files immediately. To trigger this action, run:
rake cache:invalidate
I separated this action, because it can get expensive with many assets.
Here are the tasks (add in your own IDs/KEY/BUCKET):
namespace :cache do
S3_BUCKET = 'XXXXXX'
ACCESS_KEY_ID = 'XXXXXX'
SECRET_ACCESS_KEY = 'XXXXXX'
CLOUD_DIST_ID = 'XXXXXX'
files = Rake::FileList.new(
'./public/assets/*',
'./public/assets/**/*',
)
desc "start aws"
task :aws do
AWS::S3::Base.establish_connection!(access_key_id: ACCESS_KEY_ID, secret_access_key: SECRET_ACCESS_KEY)
end
desc "compile coffee"
task :coffee => 'barista:brew' do
puts "brewed"
end
desc "compile sass"
task :compass do
`compass compile`
end
desc "run jammit"
task :jammit => [:coffee, :compass] do
`jammit`
end
desc "update s3"
task :s3 => [:jammit, :aws] do
prefix = ''
files.each do |f|
next if File.directory?(f)
key = f.gsub(/\.\/public/, prefix)
puts " upload %s" % key
attrs = {
:access => :public_read,
'Cache-Control' => 'max-age=315360000'
}
attrs[:content_type] = 'gzip' if f.include?('.gz')
AWS::S3::S3Object.store(key, File.open(f), S3_BUCKET, attrs)
end
end
desc "invalidate cloudfront"
task :invalidate do
prefix = ''
require 'rubygems'
require 'hmac'
require 'hmac-sha1'
require 'net/https'
require 'base64'
paths = ""
files.each do |f|
next if File.directory?(f)
key = f.gsub(/\.\/public/, prefix)
paths += "<Path>#{key}</Path>"
end
digest = HMAC::SHA1.new(SECRET_ACCESS_KEY)
digest << date = Time.now.utc.strftime("%a, %d %b %Y %H:%M:%S %Z")
uri = URI.parse("https://cloudfront.amazonaws.com/2010-08-01/distribution/#{CLOUD_DIST_ID}/invalidation")
req = Net::HTTP::Post.new(uri.path)
req.initialize_http_header({
'x-amz-date' => date,
'Content-Type' => 'text/xml',
'Authorization' => "AWS %s:%s" % [ACCESS_KEY_ID, Base64.encode64(digest.digest)]
})
req.body = "<InvalidationBatch>#{paths}<CallerReference>inviite_#{Time.now.utc.to_i}</CallerReference></InvalidationBatch>"
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
res = http.request(req)
puts res.code == 201 ? 'Updates' : "Failed #{res.code}"
end
end
Finally, I the Gemfile I needed:
gem 'barista', '~> 0.5.0'
gem 'compass', '~>0.10.5'
gem 'jammit'
gem 'aws-s3', :require => 'aws/s3'
gem 'ruby-hmac'
That’s it. Change as necessary.
Kicking off Inviite
Inviite is finally up and running along, getting to the point where I’m liking it. Sure, it’s only tested in webkit, and looks like crap in IE, but it’s moving slowly in the right direction. But I digress. As a first post, I suppose answering common questions is in order.
What is Inviite?
A way to send out invitations to people in the simplest way I can imagine. You don’t have to create an account (unless you want). You don’t even have to give your name/email. Perfect for anonymous meetings (I dunno - like, craigslist or something).
Then you just send out the link to people you want to attend, and other people can click whether they’re attending or not. Again, your guests can sign up if they want (via Twitter, Facebook, Google, or Inviite accounts - more to come, like Meetup and Yahoo) - or just put in their email address (so you can contact them in the future - or even just their name.
Doesn’t this already exist?
Yeah, but this is easier, fancier, and free(er). Oh, also, you can embed your inviite into other websites (like embed into Tumbler) - which people can view as details or as a calendar.
You can also pick from a few themes. Right now we have 5 - but one is The Oatmeal’s Party Gorilla - so you can’t beat that.
What else?
It’s under active development - so loads more stuff coming. Short URLs (inv.ly) were just added, so you can paste your event on Twitter or over a text message. An example: http://inv.ly/example
