2013-05-31
After working in Go for some time now, I learned how to use an unbuffered channel to build a pool of goroutines. I like this implementation better than what is implemented in this post.  That being said, this post still has value in what it describes. https://github.com/goinggo/work Introduction In my world of server development thread pooling has been the key to building robust code on the Microsoft stack.
2013-05-31
For the past 20 years I have been writing server based and application software on the Microsoft stack. First in C/C++ leveraging the Win32 API and then in C# when .Net first was released. Over the past few months I have realized that trying to build scalable code on the Microsoft stack is becoming impossible. Why, Technology and Cost!! Let’s start with the licensing. Luckily I was accepted into the Bizspark program.
2012-11-16

Two times a year the drivers team at 10gen gathers together for a face to face meeting to spend time together working on issues and setting forth our goals for the upcoming six months. In September 2012 we all converged on New York City for the second ever driver days. This time we split up into teams for a hack-a-thon. As maintainers of drivers & integrations in over a dozen different languages while we are on the same team, it isn’t often that we actually work together on the same codebase. The hack-a-thon gave us a chance to do just that. We split up into 5 teams each having members from different languages. Without further ado, here is what we came up with.

Disclaimer.. Each project currently represents exactly one evenings worth of work. Our intent is to pick the best project or two, polish them up and move them to the 10gen Labs account on github.

As with all things open source, contributions welcome.

Mongo Contributor Hub

Ever wonder what type of open source MongoDB related projects are being developed these days? We did. So we hacked together a quick Github search & explore interface for any project Github reports as associated to MongoDB! Projects are organized by language, fully searchable and sorted by followers and forks. Built with Nodejs, Express and MongoDB.

https://github.com/TylerBrock/mongo-contributor-hub

Try Aggro

With thirteen challenging questions you’ll learn the ins and outs of aggregation with MongoDB. Will you be able to complete all the challenges and become an aggregation master?

https://github.com/rozza/try-aggro

MongoDB.OData

OData is a highly used protocol with clients in .NET, Java, jquery, and many more. It makes sense to be able to support these clients with a MongoDB backend. With OData v3, the protocol is now rich enough to support the rich document model MongoDB already provides. MongoDB.OData let’s you expose your entites (MongoDB documents), complex types (MongoDB embedded documents), and collections (MongoDB arrays) via OData and includes full support for queries and OData service operations. Support for updating is almost ready.

https://github.com/craiggwilson/mongo-dotnet-odata

Slow Query Profiler

Logging slow queries is essential for any database application, and MongoDB makes doing so relatively painless with its database profiler. Unfortunately, making sense of the system.profile collection and tying its contents back to your application requires a bit more effort. The heart of mongoqp Mongo Query Profiler is a bit of map/reduce JS that aggregates those queries by their BSON skeleton (i.e. keys preserved, but values removed). With queries reduced to their bare structure, any of their statistics can be aggregated, such as average query time, index scans, counts, etc.

As big fans of Genghis, a single-file MongoDB admin app, the initial intent was to contribute a new UI with the profiler results, but one night was not enough time to wrap our heads around Backbone.js and develop the query aggregation. Instead, we whipped up a quick frontend using the Silex PHP micro-framework. With the hack day deadline no longer looming, there should be plenty of time to get this functionality ported over to Genghis. Additionally, the map/reduce JS may also show up in Tyler Brock’s mongo-hacker shell enhancement package.

While presenting mongoqp to our co-workers, we learned about Dan Crosta’s professor, which already provides many of the features we hoped to implement, such as incremental data collection. We think there is still a benefit to developing the JS innards of mongoqp and getting its functionality ported over to other projects, but I would definitely encourage you to check out professor if you’d like a stand-alone query profile viewer.

https://github.com/jmikola/mongoqp

Aggregation Pipeline Web Interface

We built a web app for the new aggregation framework. It allows you to create pipelines using a web interface, making it easy for a user to play around with the new framework without having to use the command syntax. Users can incrementally add pipeline operators to test running aggregations with different operators, and can use the easy interface as an educational tool to learn how the pipelines work. The app also allows you to pipe the results of aggregation framework jobs straight to user-defined output collections and see a history of past jobs along with their run-time. The app is built in Ruby on Rails, using the MongoMapper ODM.

https://github.com/estolfo/aggre-great

2012-10-04
fire

There are a variety of reasons businesses either do not have a disaster recovery plan or their current plan is substandard.  The beliefs of the people in charge of developing these processes (business owners and IT department) play a significant role in how effective the overall strategy will be.  This is problematic when the decision-makers have bought into one or more of the common myths surrounding disaster recovery.

Myth – Disaster Recovery is Expensive and Resource Intensive

One of the biggest reasons businesses put off developing a disaster recovery strategy is because they believe it will become too expensive and resource intensive.  As a result, they view it as more of a luxury than a necessity.  The truth is as technology continues to evolve; the costs associated with disaster recovery continue to fall.  Virtualization, standardization, and automation have all played key roles in making disaster recovery more affordable.  They have reduced the number of people required to restore systems which significantly decreases personnel costs.  In fact, a streamlined disaster recovery strategy can require only one person.  Virtualization also reduces the initial capital investment because redundant physical infrastructure is no longer necessary.

Myth – After Planning, There is No Way to Accurately Run a Test

Many businesses suffer from an ineffective disaster recovery plan and don’t even realize it.  This is because they do not believe a full-scale test is possible without significantly disrupting day-to-day activities.  Virtualization allows recovery plans to be tested against significant failures multiple times to ensure the plan is consistently effective, without disrupting day-to-day activities.  Not only are tests easy to run, but virtualization also eliminates the need to transport the IT team to multiple locations.

Myth – Creating and Managing a Disaster Recovery Solution Requires “Special” Skills

A common hesitation, especially among small businesses, is the belief that developing a disaster recovery solution requires a special set of skills.  This myth largely stems from traditional manual data recovery because end-users had to deal with duplicate, silent infrastructures across multiple sites.  Virtualization standardizes the disaster recovery process by encapsulating operating systems, applications, and servers.  This includes all the configuration data.  As a result, the entire process is now much less complex than it used to be.  Additionally, once the solution is in place it can be reliably tested and executed by staff.  Since this is all taken care of via automated processes, no special skills are required.

Myth – Disaster Recovery Plans Are like an Insurance Policy that Never Gets Used

The most common myths which prevent businesses from taking disaster recovery seriously are that it is unnecessary.  They believe disaster recovery is a sunk cost similar to a car insurance policy of someone who never gets an accident.  Even if a disaster never happens, the recovery plan still provides a variety of benefits to the business.  One of the most common uses is as a migration plan template anytime a business switches data centers.  Depending upon the industry, disaster recovery plans may also be a compliance requirement. In order to accurately assess the need for a new or improved disaster recovery plan these myths must be eliminated from a business’s belief structure.  As long as these myths remain, the odds of successfully implementing an efficient disaster recovery strategy are slim. Tony Spinks believes every company should consider using Disaster recovery services.  These services are critical for keeping businesses running after system failures.  Data Centers offering disaster recovery services provide the redundancy.

2012-09-24
security

The protection of your intellectual property, or any information that you post to your blog, forum or website, can be a tricky subject. While there are those unscrupulous few out there in the cyber world who will purposely copy your words as their own, there are more people that are simply ignorant of the laws, unaware that they are stealing when they copy and reuse your musings. According to a criminal lawyer at an Orlando based firm that we spoke to, there are steps that you can take to make sure that your intellectual property is protected. Here’s what you can do:

1. Obfuscate Your Code

To protect yourself against thieves, use software like Dotfuscator and JavaScript Obfuscator. These types of software programs will obfuscate your coding, making your content more difficult to steal by making a program’s source code difficult to read by humans. When someone tries to copy and paste your blog entry, they’ll be left with unreadable, useless coding. Other helpful software includes vGuard, Jasob and Salamander .NET Obfuscator.

2. Post a Permissions Policy

A page of your website or blog should be dedicated to an explicit permissions policy. This policy will tell others what they can and can’t do with the content that you have posted. The key to a great permissions policy is clarity; people should know exactly what it is that they can do without your permission and when they’ll need to contact you. Once you’ve published your policy, you’ll have something to point to if someone violates it.

3. Contact the Violator

If you find that your content has been used by someone else, the first thing to do is to send a polite email to the violator. 99 times out of 100, the person who used your content has no idea that they’ve done anything wrong. Kindly thank them for finding your content interesting enough to use on their own site, point to your permissions policy, and offer suggestions as to how the person can use your content legally.

4. Demand Removal

If you’ve found the one person with whom a polite email doesn’t work, you’ll need to escalate your efforts. Sit down and draft an email demanding that the offender remove your content from their site immediately. While it’s still suggested that you remain tactful and polite, you may want to be a bit more forceful. Direct the offender to your permissions policy again, letting them know that they are violating copyright laws by allowing your content to remain public on their page.

5. Contact the Hosting Site

If steps two and three haven’t worked with the violator, you’ll need to contact their hosting site. Find a site online that will allow you to search for the person’s domain registration and, once you do, send an email to the hosting service. Let the hosting service know the steps you have taken and that you are now requesting removal of the web site. The powers-that-be at the hosting site will investigate your claim and, if it is found to be with merit, will take the site down.

6. Get Legal Help

At the end of the day, your last bit of recourse may be found in hiring an attorney. A criminal defense lawyer can go to bat for you, helping you to have the offending content removed from cyberspace.

If you find that someone has copied your content for their own blog, be flattered; it means that they’ve found your content useful or relevant. Remember that most people don’t understand that they are violating any laws, and an email is often enough to get your content removed from the offending website. If you follow the steps above, you’ll ensure that your intellectual property remains yours.

Katie Hewatt is a legal researcher and contributing author for the Florida Law Firm of Katz & Phillips, which deals with Internet crime cases. The Orlando law firm keeps up to date with the latest technology online and the ever changing cyber crimes involved with it.

2012-09-17

Steve Francia at OSCON

At OSCON 2012 in Portland I gave a presentation on building your first MongoDB application. Over 150 people were in the audience, a pretty significant number of this type of hands on tutorial. Certainly worth the weeks of preparation that went into developing it. While at OSCON I put the slides online at SlideShare where during the four day conference the amassed over 20k views and within a couple weeks over 30k views. Within a month it had been viewed more than ten times the total attendees at OSCON, one of the largest technical conferences in the world.

How was this presentation so successful that it amassed more views in it’s first week than any other presentation had accumulated over 12 years? Here are the four critical things I did to generate such an amazingly high number of views.

Have an interesting & unique topic

The first two items are focused on the preparation. Your topic needs to be something that people are both interested in and find interesting. To be both it should be unique about something popular. I’m really passionate about building great things and so are a lot of people. Like many people out there I’ve looked into using new technologies to make the process of building applications easier and more enjoyable. This behavior led me to explore NoSQL solutions which led me to MongoDB. MongoDB is one of the most popular technologies right now. It’s claiming a significant lead over other NoSQL technologies and catching quickly to the leaders in the very established relational space. Remember I’m commenting here on popularity and interest. This is easily discoverable by looking at things like google trends or frequency of mentions on niche sites like hacker news. With popularity comes noise. To stand out from the noise you need a unique topic or take on it. In giving this presentation I chose to focus on ‘writing your first ruby application backed by MongoDB’. From skeleton to deployment. To my knowledge this hadn’t been done before and was something that stood out to people.

Make it broad and approachable

To reach the largest group of people you need to appeal to a large group of people. This means giving a presentation that is approachable that many different kinds of interested people could follow along and find value in. Given that this presentation was aimed at first time MongoDB users without much development experience it really resonated with a lot of people. One nice thing about OSCON is that it attracts a really diverse crowd. If  it was appropriate for the audience there, it would work well for the broader Internet audience which OSCON is a microcosm of.

Cater to the online audience

Whenever I create my slide decks I’m always focused on two audiences. The live audience that will see the presentation projected onto a screen and the remote audience that will be viewing it on slideshare via their computers/laptops/tablets etc. Most people ignore that second group, but a good presentation will always reach more people online than in person. If anything it’s the group that can’t afford to be ignored. I ensure that the slides flow well even without my narration. I give them to a few friends without the accompanying talking and see how well they can follow them. I make sure to use good layouts that use the space well and are readable when on the projector from the back of the room as well as when embedded on a web page. It’s important when using slideshare and the other sites that you review the presentation on them as well as embedded as conversion is sometimes destructive, especially if you are using a unsupported font.

Give it a good start

Success begets success. During the live presentation I put the slides up on slideshare so that the attendees could follow along at their own pace. Since the audience was largely working with the code in the presentation it made a lot of sense for them to bring it up on their laptops. With the 150+ audience members all viewing the presentation that gave it a very big initial push. A few of them also tweeted it which helped even more people to view and follow it. Slideshare has a ranking algorithm that tracks when a lot of people view a presentation in a short amount of time and flag it as a “hot” presentation. Within a couple of hours it had 600+ views and a couple dozen tweets on twitter.

That evening I got an email from slide share that my presentation was getting a lot of views. I got a second email that it was experiencing a high volume of tweets as well. It continued to climb up the ranks. The next morning it was selected as the SlideShare presentation of the day.

@slideshare: '#OSCON 2012 MongoDB Tutorial' by @spf13 is a #SlideShare presentation of the day http://t.co/74DqqsJR cc @oscon on Twitter http://twitter.com/slideshare/status/225393208581038080"

On Tuesday it had amassed an outstanding 7k+ views

From there it became a presentation of the week. It was featured prominently on the home page where it remained for the next 3 days at which point over 22k people had viewed it.

Once it fell off the home page and popular/week lists and tweets became less regular the number of views were still significant, but had slowed to a few hundred a day. At the time of this writing it’s over 30k views and climbing.

See also my post on “How to deliver a great conference tutorial” where I walk though how to prepare and deliver an effective conference tutorial.