Friday, August 15, 2014

The C# Coding Guidelines are now open-source!

The ability for an organisation to request the original Microsoft Word files and adapt them for internal use has always been an important recommendation of the C# Coding Guidelines. Since I first published the PDFs on CodePlex in 2009, I have received numerous requests for that, but the process has always been a bit cumbersome. Hence my decision to convert the guidelines into MarkDown and to push them to a new GitHub repository.

The corresponding license, based on one of the template licenses provided by Creative Commons, allows you to fork, adapt and distribute that modified version within your organization as long as you refer back to the original version on GitHub. It's not required, but you would make me a very happy man if you credit me as the original author. And if you have any great ideas, recommendations or corrections, either submit an issue, or even better, fork the repository and provide me with a pull request.

Notice that I decided to use some of the features provided by Markdown Extra because that allows me a bit more control on the generated PDF that I will ultimately publish back on the current landing page on CodePlex. I know that GitHub will not render everything correctly and page breaking in the generated PDF sucks, but that's the closest I could get to the original version.
Some of the things I'm planning or considering:
  • Splitting the guidelines into two separate documents; one on coding practices and one on design practices.
  • Introducing some of the guidelines proposed by Object Calisthenics.
  • To setup a new landing page through GitHub's pages and drop CodePlex completely.
  • Convert the cheatsheet into a Markdown file.
  • Start evaluating the new C# 6.0 language features.
  • Update the proposed Static Code Analysis rule sets for Visual Studio 2012 and 2013.
  • Verify the ReSharper style sets
  • Evaluate ReSharper's pattern matching expressions to see if they can be used to detect a part of the guidelines. 
So if you have any ideas, go fork my repository and send me some nice pull requests…

Tuesday, July 29, 2014

Universal Apps, WP8.1, xUnit2 or what else is new in Fluent Assertions 3.1

Yesterday, much later than planned, Fluent Assertions 3.1 as well as its companion project Fluent Assertions for Xamarin were released on Github and NuGet. As is becoming quite common this year, the honors for most active contributors once again go to Adam Voss and Oren Novotny. Next to being the author of the Xamarin version of FA, Oren contributed the necessary internal redesign to support Univeral Apps, Windows 8.1, Windows Phone Silverlight 8.1 and Windows Phone 8.1. But as if he didn't do enough, he also found the time to add support for xUnit 2. At the same time, Adam has been instrumental in solving the many little bugs reported through Github as well as refactoring and improving a lot of the code related to the structural equivalancy assertion API. Another contributor, Kevin Kyszyk, introduced a syntax to assert that a particular assembly doesn't reference another one.

I've focused a considerable portion of my free time on rewriting the internal detection logic that is used to connect the framework-specific parts into the core Portable Class Library. My original assumption that sooner or later somebody would touch the AssertionExtensions class was wrong. I'm now borrowing the detection logic from the Portable Class Library Contrib project which uses assembly scanning for linking the PCL and non-PCL assemblies together.

A nice side-effect of Oren's usage of Shared Projects to remove any duplication between the framework-specific assemblies, that I could reintroduce support for the plain old .NET 4.0. During the conversion to a Portable Class Library I discovered that .NET 4.0's PCL implementation did not support the XML classes yet. That's why I had to retarget the PCL project to .NET 4.0.3 At that time, I underestimated the impact this would have on existing developers. With v3.1, I've fixed that.

As usual, the detailed release notes as well as the zipped binaries can be found here. If you're interested in future builds you might be interested in the MyGet feed. Our development pipeline will automatically publish a new NuGet package for every successful build.

Happy testing!

Thursday, June 19, 2014

QCon New York Day 3: Humans and engineers, Career Development, Remote Teams and Culture

Building a NoSQL database

Man, my brain still hurts from the first two days, especially around all the clarities of micro-services. But, longing for more information, day 3 started with another keynote. Not as good as yesterday's, but the story about how Amazon build their own NoSql database contained some nice lessons. For starters, they insisted on planning for both scalability as well as failure. The former is usually something that doesn't get immediate attention, but that's usually still more than what we do with the latter. And again, Netflix is being mentioned since they invented Chaos Monkey and Latency Monkey. Another thing that I do agree with is consistent performance. How many companies actually monitor performance day by day? In my experience dev teams only start addressing performance when there's an immediate issue. One interesting concept they introduced is the Blast Radius. During their failure testing they explicitly analyze how far that failure reaches into the application landscape. With that information, they've tried to tune the interfaces between components to become a bit more resilient.

WP_20140614_11_21_36_Pro

Mentoring engineers

To my surprise there was an actual TypeScript session in this slot, but the title of another session -- Mentoring Humans and Engineers -- sounded too interesting. Daniel Doubrovkine showed us how he approaches mentoring new employees within his current organization. They've based their approach on his experience that getting the strongest people at any level and keeping them is the single hardest thing in any organization.

This all starts by promising an interesting learning experience, both on the job as well as outside the job. For instance, new employees are also shown around in New York. What places they should visit. Where to find a nice place to stay. How to meet-up with new people. Next to that, they pick their mentors from the most experienced, pragmatic AND patient people that understand starting with clear milestones and then gradually moving to goals is the best. They know that fostering ownership and trust is essential, the room for self-exploration is required, while not overwhelming them with both information and too much freedom.

Even more important (and something I often forget in my desire to help teams) is that failing is okay, as long as you learn from it. You should really create an environment that provides feedback, but take responsibility to protect them from anything that would de-motivate them to try new things in the future. At the same time, teach them how to fail by given them an assignment that is difficult or impossible to do. And don't forget to advertise them within your organization. Always introduce them to whoever you run into and openly praise them for any accomplishments. Daniel's closing statement was particularly interesting since he warned us not to underestimate a young new colleague. They might be much smarter than you are….

Climbing off the ladder, before we fall off

Spotify has been a big example for us, especially through Henrik Kniberg's excellent book Lean from the Trunches. And QCon wouldn't be QCon if they would not have a Spotify session. Chris Angove, a chapter lead from Spotify New York explained how he tried to find the balance between alignment and autonomy (or "do what I say" vs "do whatever"). They quickly discovered that this is destined to fail, simply because these two ends should be orthogonal. Instead of that, they tell their engineers what to do, but don't tell them how to do that. Somehow this freedom brings along a lot of responsibilities. To help the teams know their part in the big picture, they can give them missions, short-term goals and a product strategy.

This is all fine and well, but after more than two days of sessions, this is not what sets them apart from the rest. But what did is the way they approach career development. Traditional companies generally use a linear ladder where a software engineer can potentially climb to senior software, architect or even CTO. This provides a clear structure for promotion, and thus more salary and status. However, according to Chris this model can also be a factory to eject people due to limited management positions or by promoting people beyond their capabilities. In short, it provides simplicity for the manager, not the engineer.

Alternatives exists. For instance, some companies use two ladders; one for management positions and one for software engineering positions. Although it provides a technological track, and it clearly sets up easy ways to recognize accomplishments, it still looks like a linear ladder. Similarly to the previous model it assumes that the only way to grow is to take more responsibility or control. It still doesn't answer how to experiment or switch between roles.

Spotify believes that people that add value should be paid more money regardless of their role or position. To support that they stimulate engineers to try something new, which doesn't necessarily mean they stop what they are doing. In other words, they try to get them out of their comfort zone, to acquire a new skill and to push themselves in a new direction. More specifically, to shake things up a bit and see what talents or skills somebody really has. So instead of this rather rigid ladder, they've introduced add-ons that add both personal value as well as business value aligned with the skill-set and interests chosen by the engineer. In a sense it is engineer-driven but supported by company. The manager must work with the engineer to help him or her to find those challenges or even define a completely new add-on, or even giving them some time off to attend trainings, sessions or workshops. And those add-ons don't have to exist upfront. It is perfectly fine for an engineer to come up with another add-on that adds value to the company. Some the add-ons currently being tried at Spotify include a speaker, a road manager, coach, evangelist, mentor, trainer, writer or even open sourcer. Even an architect is treated as an add-on, simply because they help in alignment rather than to make decisions. If you ask me, this sounds like a great model.

Interviewing for culture

Another slot in this awesome conference with an open-space session on culture. This included a great discussion on mentoring and how important this is for your career. Several of the participants shared their opinions on how a junior engineer needs to be taught on how to get the most out of this skills and how to advertize him- or herself within an organization.

Another discussion was about how to determine if somebody is going to match your organization's culture during an interview. Inviting somebody for lunch helps to see how somebody behaves in a new group was one suggestion. Having full-day interviews to allow the person to relax was another. However, one participant said they stopped doing that because candidates would be completely drained at the end of the day. Another participant told us that upon arrival of the candidate, they first asked about there private life, kids, etc and then came back after 5 minutes or resulting in a much more relaxed interview. Doing code assignments from home is also very popular, but some mentioned they were worried the candidate would game that system.

The final topic dealt with introvert colleagues. I explained how something like Flowdock seems to help those people that have great ideas, but just don't feel comfortable enough to share that in an open discussion.

WP_20140614_15_43_48_Pro

Remotely working at Github

Last year's talk by Github employee Zach Holman already showed many of the merits of that company, and this year Matthew McCullough continued by elaborating on some of the things they do to foster remote working. I already knew that Github employees can spend 20% of their time on innovations, learning new things, but I didn't know that on average 70% of the people work remotely. That explains the amount of energy they put on building tools and offices that support that.

For instance, they've build an internal app that provides a central location for discussions, sharing ideas, their internal communications handbooks and having live video chats. They also organize mini-summits where two engineers can do an internal talk which is then shared through that same app. Even cooler, they build tools to allow them to listen to the same music all over the world. So if somebody makes a remark about a cool song at the other side of the world, it's almost as if they work at the same office.

They also pro-actively build small offices with a living room or café-styled look as well at different places in the world to allow people to have a nice place to meet informally. Those offices also have some kind of phone booths to work in In fact, to better understand how it is to live outside of the US and in a different time zone and to work on a global project, the CEO decided to move to Paris for a year or so. They also organize beer:30 meetings where teams have informal virtual meetings with other global teams.

While discussing the communications guide, Matthew shared some of the foundational principles Github. First of all, teams make decisions about what they do, not committees of carefully selected people. Egos are not allowed in discussions. In the end, the best argument wins. That's an interesting statement, especially considering my own challenges to give somebody enough room to make their point sometimes. But even more striking is that everything is open within Github. The financial situations, salaries, expenses and even meetings that deal with the future and strategy of the company.

That's definitely something I have never seen before and from which a lot of organizations can learn from…

Friday, June 13, 2014

QCon New York Day 2: Freeing people, imperfect architecture, lean coffee and unanswered questions

That was the weirdest conference party ever. No music, no dancing, just free food and beer. Well, our of me, two Austrian guys and a fellow from Dakota had plenty of great stories to share. And did I mention the free beer?

clip_image001

Freeing people and optimizing the tools

As any good professional would do, I also was at the conference venue on time, just like anybody else. And for once, the keynote was worth my time. Dianne Marsh, director of engineering at Netflix shared how Netflix ensures their engineers get the freedom to innovate. Their basic vision is to let free of the people and to optimize the tools. They do that by having their managers provide context rather than control the people and try to attract and retain talent. Obviously engineers have the tendency to go any which way they can, so managers are supposed to help them to have some kind of alignment.

They also should stimulate experimentation in independent paths of exploration. If you've been attending the Lean Enterprise talk from Trevor Owens you should understand why that's such a great thing to do. Next to that they heavily invests in continuous delivery. Developers can test and deploy their own code using a fully automated deployment pipeline. If this involves larger and more risky changes, they do what they call a red/black deployment or canary testing. In other words, they run the two versions side-by-side and gradually move our users from the old to new environment.

One other thing that caught my attention is how they approach software development as something that can fail at any time. That’s why they created the Chaos Monkey that will shut down random Amazon VMs to see how the system behaves and the Latency Monkey that will cause timeouts in network connections at random times. This is pretty special considering how often that day I heard references to that in architectural discussions.

In all seriousness

Michael Feathers is one of my hero speakers, not only because of his excellent book, but also because most of his talks are really inspiring. He has spoken about Conway's Law numerous times, but this was the first one where he used this law to propose organizational solutions. Roughly he said that irrespective of the greatness of your architecture and all the agile practices your applying, sooner or later you'll run into scaling issues caused by the organizational structure. I've been working at a client that grew from 8 to 120 people in about three years and Conway's Law is really visible there. It starts to get particular difficult when you can't get all the developers on the same floor. That's one of the reasons Michael has a great interest in the whole micro-services hype. Since each micro-services can be maintained by at most one team, it provides not only architectural scalability but also on the organizational level.

Embrace Imperfection

Right after Michael, I went to Eric Evans' talk on embracing imperfection. Last year I left his full-day tutorial on Domain-Driven Design because it was too boring, but this session was absolutely great. After admitting he wasn't really sure what architecture really is (!), he talked on our failed attempts to build robust and resilient systems. That's why he believes Netflix is doing such a great job. They've essentially managed to reach perfection in handling imperfection. As a long-term architect that will profoundly affect my future endeavors.

Provided that I understood him correctly, his general rule-of-thumb is that within a bounded context (BC) you should have 90% of the functionality covered by an elegant architecture. The remainder doesn't have to be so perfect, as long as the BC is aligned with the team structure. That ensures that any shortcuts that are taking are well-known within the team and won't surprise other teams. By further decoupling the BCs from each other using special interchange contexts that translate messages between BCs (aka the Anti-corruption Layer), and not sharing any resources such as databases, you can localize the imperfections. Oh, and with that statement of 'not being so perfect', he actually meant to use if-then-else constructs rather than well-designed polymorphostic abstractions!

clip_image002

More Eric and Michael

As if this day wasn't already great enough with individual sessions, wouldn't it be cool to have the both of them open up a panel discussion on architecture? Well, here you go. In fact, that panel was joined by Leo Gorodinski who used Greg Young's EventStore to build a system in F# and Duncan DeVore who build an Event Sourcing system in Java. Yes, you've heard that right. They were actually discussing Event Sourcing as an architecture principle in the context of micros-services at Qcon. After the session I talked a bit more about that with Leo and Duncan and they told me that Event Sourcing is only recently getting recognition outside the .NET world. Guess I should be proposing a talk on distributed occasionally connected systems using Event Sourcing for next year then. Anyway, one question that was discussed quite extensive is whether domain modeling, or more specific, aggregate boundaries are still useful while developing micro-services. I couldn't judge the later, but in the context of event sourcing I shared my experience that having proper aggregates allowed us to move from a traditional relation database system to Event Sourcing without too much of a hassle. However, I also shared that Event Sourcing is not a silver bullet and you should be prepared to the need to learn a lot of new technical and functional concepts.

Lean Coffee

To stay within the context of the architecture panel discussion, I attended a Lean Coffee session on improving architecture. Lean Coffee is a simplified version of Open Space where people can propose topics that are openly discussed for a maximum of 5 minutes in the order defined by dot voting. I even asked Eric, Leo and Duncan to stick around for this talk because there were a lot of questions after their session.

clip_image003

Since 5 minutes is short, we covered several topics:

How to handle exceptions in an event-based architecture; After a short discussion we learned the attendee meant a command-based architecture. Somebody proposed to have a callback REST API that command handlers can report errors to. Others proposed to associate each command/event with a correlation ID and have some framework in place for registering errors. I myself proposed to validate the command for correctness synchronously and handle any business errors asynchronously through a centralized registry, but keeping the errors really functional.

How to compose micro-services from aggregates and how to select the right size and maintaining micro-services; Talking about awkward silence, this was one. Nobody in the group had the experience nor got anything from the QCon sessions that could answer this. Bummer…

What else are people talking about EXCEPT micro-services; Since I'm working with an Event Sourcing architecture and this was also discussed in the previous panel discussion I shared some of our trials and tribulations of moving from a traditional architecture to ES.

Usual stuff you do to improve architecture and how to visualize and/or measure this; Most of us mentioned the SOLID principles, reducing technical debt, pair programming and code reviews, but we couldn't entirely agree on DRY. Some of us thought DRY within a component or bounded context is important but outside that boundary is not too important. This especially aligns well with Conway's Law because preventing duplication between teams is a lot more difficult or even undesirable. However, not everybody agreed with that.

Six questions on micro-services I hoped to get answered, but didn't

So after two days of architecture sessions and discussions, I still have a lot of questions unanswered…

  • How to group aggregates in micro-services?
  • How do you prevent a micro-services from growing too quickly and becoming macro?
  • Should micro-services call eachother or should they publish events only?
  • How do you version messages to be both backwards compatible as well as forwards compatible?
  • How do you manage the dependencies between the services?
  • How do you effectively aggregate data from multiple services to prevent N+1 calls?

Anybody??

Thursday, June 12, 2014

QCon New York Day 1: Micro-services, Team Latency and the Lean Enterprise

So now that I've got the unofficial part of this trip behind me and I managed to get through a very in-depth full day workshop on reactive extensions in JavaScript with my limited JavaScript experience, it is time for the main part of the conference. As is customary at QCon, each day always starts with a pretty boring introduction of all tracks by the track owners. I'm honestly wondering why they force those people to get on stage for such a large group. They are obviously very uncomfortable with it. Even the guy that opens up the day sounds like he doesn't actually want to talk about QCon.

clip_image001

Dart

Anyway, Gilad Bracha, Google employee and author of Newspeak brought us up-to-speed on web programming in the future. He promised not to make it too much of a marketing talk. But after several (not so succesful) demos on Dart he horribly failed on that promise. Why not invite Anders Hejlsberg and let him talk about TypeScript instead. That is a speaker with excellent speaking skills, has charisma and can tell a story.

Flight

In my current project we have historically used KnockoutJs, but we are in process of switching to AngularJS for the newer features. Twitter has been sharing their internal framework called Flight, whereas Facebook is actively pushing React, a JavaScript framework based on the concept of a virtual-DOM. To get a better understanding of the first, I decided to go to Kassandra Perch's (unexpectingly short) introduction. Well, either I didn't pay enough attention, or my JavaScript experience really is hampering me, but I totally didn't get what she was doing. I might be wrong, but those JS classes looked like she was coupling the DOM and the UI behavior within the same code. Call me a purist, but avoiding this is what EmberJS, AngularJS and KnockoutJs have been doing all along.

Oh, and one of the guys in the audience asked her how to select the appropriate framework for your project. She proposed to create a nice table and do a feature comparison of the various frameworks in the context of your project. I think that is stupid. You're not going to understand a framework's limitations if you don't actually use it in a real-life scenario. Just do a spike and see if you can rebuild an existing page using the framework of your choice.

Accidental architecture

Historically, the Open Space concept has been one of the best aspects of a QCon conference, so I decided to attend one about architectures you always wanted to learn about. Its painful to see how many people are facing the same problems over and over again, and how few opportunities you have to share experiences. Somebody was proposing the topic of accidental architecture but every a short discussion she actually concluded that they didn't have a known architecture at all and hardly any process. Each developer took a story, wandered off and did their own thing. In my own experience, authority and rules are not going to work, so I proposed to using an open space within the organization to find out what problems the developers themselves were facing. If they see the problems themselves, they are more likely to try to fix it themselves as well.

clip_image002

Micro-services

I also talked with somebody who has like a 100 micro-services running in production and is suffering from operational issues and data consistency problems. They were trying to solve the event up conversion the same way as we do in our Event Sourcing architecture and threw in all kinds of enterprisy products to help them. As far as I can judge, they were far ahead of what I've heard at QCon. We had a nice discussion and we both concluded that their next challenge is to accept that it will be impossible to keep all the micro-services in sync in terms of message and API versioning and that they should invest heavily in pro-active signaling and continuous delivery of those services.

More micro-services

Talking about micro-services, I also attended a talk by Yoni Goldberg on how Gilt, a US fashion outlet site, decided to break their monolithic website into about 50 micro-services each owned by a separate team. They used Docker to ease deployment of those services on shared Amazon VMs. According to the speaker they were very successful in that migration, but were now facing the more operational challenges of supporting so many dependent services. Apparently he has taken the same conclusion that tooling for supporting that, as well as continuous deployment, are becoming more important by the year. He concluded the talk with the advice that if you consider moving to micro-services, just start building every new feature as a micro-service and add the required tooling along the way.

Anyway, I'm not yet convinced about this micro-services thing. It solves certain problems quite nicely and is very scalable, but it also introduces a lot of new challenges around API/message versioning, finding the right bounded context and operational support such as monitoring and pro-active signaling. For now, nobody has really solved this entirely.

Team latency

Edmund Jorgensen delivered a refreshingly fun session on the latency of teams and how bringing down latency is best thing an organization can do. So many managers are focusing on the short-term needs rather than understanding how expensive any form of latency is. Think of technical debt, manual deployment steps, or anything else that prevents a team for speeding up. His basic approach is that reducing latency creates information earlier and early information is worth a lot of money. In fact, it saves money, so why do manager still persist on spending all their capacity on building features rather than having a good balance between functionality and reducing latency. Somehow, during his talk I had this sense of a déjà vu, like I've had heard the exact same story before. And then I realized that he was actually the author of this excellent article on speeding up your engineering organizations. Go make sure you read that.

The Lean Enterprise

The final session of the day was delivered by Trevor Owens, the author of Lean Enterprise. He showed us that a lot of the current successful startups were founded by prior Google developers. So why did they leave and became so successful? Well, according to his findings, using equity instead of salary as an incentive really gets people to innovate while most organizations have a rather traditional reward system. He also used a lot of fun facts from the world of the venture capitalists and how they never bet on a single horse. In fact, their entire investment strategy is based on the approach that one of the investments is going to be paying the losses made by the others. So he advices organizations to have multiple teams work on innovation at the same time. Or even better, let them leave the company, do their innovations autonomously and then buy them back, just like Google did. I'm definitely going to read that book.

I also attended the evening keynote prior to the conference party where Peter Wang tried to get us to focus on the bigger problems like early innovators did rather than on what technology stack to use. He's a great speaker, but by then my brain was already starting to shut down. Attending in-depth QCon sessions from 9 until 19 is not for the faint of heart.