We would fail to write bubble sort on a whiteboard… Should we be proud?

Recently there was very interesting activity on the Twitter where famous programmers just admitted they are very bad at Computer Science (CS) and other fundamental concepts and still doing very well.

Now, this is very interesting. To me, this is just another manifestation, that knowledge has a little value today. Everything can be easily Googled in a very short time. Yes, the search takes some time, but it is much more efficient than trying to learn and remember all the details of CS and technology concepts. And I also admit: I would fail to write bubble sort on a whiteboard and estimate my algorithms’ complexity while being a successful developer for more than a decade now.

But one may come to the (I believe wrong) conclusion that it is ok to be unable to estimate the algorithm’s complexity and not to understand the essence of fundamental algorithms and their complexities. Algorithms are the methods for solving problems of procedural nature while design patterns are the methods to solve software structural problems and I would argue both are the very basics of software engineering. If you do not know these methods, when you are tackling the tough problem, you basically don’t know what you don’t know. You are unable to discover that your problem can be reduced to one of those fundamental problems and there is already an elegant solution to it. I definitely see my ignorance in CS as a problem and I am currently deliberately studying algorithms :-). And also clean code, which I always enjoy learning and practicing.

Here is what John Sonmez experience was after having learned algorithms:

All of a sudden it was like I put on special glasses that let me see the world in a different light and all these problems, all these places where I was like there’s nowhere in the real world where I’m going to use algorithms in my code, bam, it was popping up everywhere. It’s like, “Whoa! Look! Oh, I recognize this. This is like a min-max problem.” Bam! All of these places I started writing really efficient, really good code because I could see the problems.

I can also see the pain when job interviewers start asking puzzles and algorithms stuff. I hate it. If you want a job you are stressed already and now you have to focus, concentrate and tackle the tough problem at the blackboard as if you would do it at your real work! Problem-solving is very creative activity and It can not be done effectively under stress because the brain actually focuses on security aspect first and foremost. I don’t know how to replace this practice, but it is total evil to me. Interviewers, please, don’t do it to us, developers, unless the job requires this type of skill and knowledge. Pretty interesting article on the related topic from Yegor Bugayenko. But since interviewers still continue asking, maybe realizing the importance of fundamentals, we should be prepared!

Hence, if you have some spare time and you have a choice either to learn some new cool JavaScript framework, which will be forgotten in few years or some of the algorithms or design patterns which will serve you throughout all your career, I believe it is better to select last option.


Few guidelines on unit testing derived from my experience

When I first started using unit testing in my practice, I had no idea what I was doing. I was also frustrated. It made little sense to me to have everything unit tested because writing tests and preparing test data took me too much time.

Ever felt the same way? Don’t understand where and when to write test and where you should leave the code alone? Don’t worry, this is normal. This happens to everyone who tries to improve the way he/she does software. I can’t give you a concrete prescription, no one can, but in this post, I will share what worked for me in terms of unit/automated testing. This will, probably, work for you.

Don’t write unit tests for simple cases. One objective way to measure the return on investment (ROI) from unit tests is to measure how much time they save for development team by catching regressions. In simple cases, when the code is not going to be changed or code is pretty straightforward it is unlikely that you will get regressions, therefore it is likely that you will have no ROI at all from your unit tests. Furthermore, you will need to maintain them. The law of diminishing returns works here. You can get 80% of the benefit from covering only 20% of your code with tests. This 20% of code contain your core business logic, which delivers the most value to your customers. Everything else is some kind of glue code, configurations, mappings, frameworks, libraries interoperations and so on. The more effort you put to cover this code, the less ROI you will have.

Do large acceptance tests for refactoring. If you plan to do some large refactoring or restructuring, classical unit tests will not help. In fact, they will go into your way. Classical unit tests are small and test some small parts of the system. When you start changing things, unit tests start glowing red and you will need to delete them. The large acceptance test captures the whole business case of interaction between your system and the user. Such a business case is something that brings real value to the business and should not be changed during refactoring. Relying on acceptance tests will increase your chances to refactor without damage to the business. In developeronfire podcast, Nick Gauthier (episode 149) reported that he had his largest success in the career by moving the web application he worked on from classical client-server architecture, where HTML was rendered on the server, to the single page application (SPA). Acceptance tests made transition really smooth for his team. My refactoring team at SimCorp also had a big success of not jeopardizing our product’s quality by major refactoring we did. That refactoring touched almost every user screen in the system. My team lead insisted on having large acceptance tests suite, which eventually ensured our success.

Unit test complex algorithmic stuff by classical unit tests. As you probably know there is classical and London school of TDD. According to the classical school, unit test just applies input data to the system under test, harvest the results and compare them to expected results. According to the London school, unit test invokes system under test and then checks if it behaves as expected, i.e. it calls its collaborators correctly and in the correct order. While I feel frustration from unit testing simple cases, I get a lot of value from classical TDD when I develop complex algorithms. Value, in this case, comes from regressions which happen during initial development, because when you develop a complex piece of software you can potentially spend days in one unit trying to put things together. I vividly remember one programming exercise from my SimCorp days. I had to develop a program, which would take APL data structure of any complexity (for instance matrix of matrices) and generate APL code which would recreate this particular data structure. My first attempt failed because after 4 ours of work I was far from being done. And most of this time was spent on retesting program operation with different types of inputs after every change to the algorithm. Next day, I have tried classical TDD. And the process was not only fun but also fruitful. In about 4 hours I was done with approximately 30 tests. I remember my impression was that I would not have finished such a program in this amount of time and confidence without unit tests.

Apply London school TDD to the integration code. What if your core business logic is not algorithmic one? What if you bring the value on the table by integrating different systems together? For a very long time, it was an open question for me. In such a cases I still want to be sure my core code is tested well. However, classical tests are awkward because integrators often don’t even have inputs or outputs. I believe London school tests are perfect for them. Once, at StrategicVision, I had to develop a system that would download videos from video hosting services, extract audio from those videos, transcribe audios and finally save transcriptions and video links to the database. No business logic in a classical sense, right? My code just invoked video hosting web service, then invoked downloader, then invoked transcription web service and finally invoked repository to store results. I wrote a bunch of tests which were testing, for instance, such facts: if the system under test invoked downloader for a particular video, it should later invoke clean up for this video; if the system under test invoked database repository to store results, before that it should invoke transcription web service.

These guidelines are highly subjective of course, but at least they work for me, at least at this point in my career. Hopefully, you will also find them helpful.


Refactoring disappointments

During my university days, I was a big fan of refactoring. I believed it should be done as much as possible in every situation when you encounter a code smell. But university days meant a lot of books and no real practice. Migration to the industry, specifically to SimCorp taught me that my beliefs were not true. Refactoring can be harmful in certain situations and it also can be really disappointing and demotivating activity. Here are some cases which I experienced in my career:

Refactor-distractor. Refactoring definitely takes away your time which could have been used for development of new features or fixing bugs. Now, sometimes refactoring is worth doing when you can clearly see the benefit of it when you can get 80% of improvement with 20% of the efforts. For example, you can see that few hours of refactoring can help putting important functionality under test or improve understandability of the frequently changed code. But often you will find yourself doing code improvements on the part of the system which rarely gets changed or doing improvements to the code which is already good enough. Diminishing returns law works here. You are pushing towards better code but into every further code improvement, you need to put more and more resources. I don’t know about you, but when I got caught by this type of refactoring, at the end the day I always feel pissed. The feature I am expected to deliver is not progressed and resulted code is not ideal because it is just never ideal.

Refactoring with no domain knowledge. Avoid doing refactoring if you don’t understand the domain of the software very well. This lesson was delivered to me by SimCorp. They have extremely complex financial software which they produce for a very long time. Every system which is in production for a long time has a lot of legacy code, SimCorp Dimension is no exception. And here I am, a freshman developer who just defended Ph.D. on refactoring and given the task to fix some bug deeply in the core of business logic. I discovered a function which length was over 7 hundred lines. It was definitely bleeding for refactoring and I immediately jumped in and starting extracting new functions. But there was a daunting problem! I did not know how to name those new functions. I had the best intentions and good technical knowledge but these prerequisites were far from enough to be able to do worth refactoring. I was running around distracting my colleagues and asking what they thought was the best name for this or that piece of code. I have finally done with this refactoring. It took few days and I was already exhausted and disappointed that my bug fix took that much time. But this was only the beginning of the trouble. Now, when something went wrong with this code, people started redirecting bug reports to me, although I absolutely was not aware what this code did in terms of business logic. No one actually came to me and said thanks for this clearing up! And this is because they were not happy with my changes at all. They knew the code well and now I have changed it and gave awkward names to the functions. Without domain knowledge, I made things even worth. Even after almost 4 years of my work there my, now good friends and colleagues, joked about names I gave to extracted functions.

Large refactoring by a dedicated team. As I said I was huge refactoring fan when I started at SimCorp. Therefore I joined the refactoring team which was forming soon after I joined this great company. The task of the team was to improve very badly rotted part of the system. Although I learned some good lessons from this experience, in about a year I understood it was a bad idea to have dedicated refactoring team. My discovery was supported by plenty of blog posts and books which described similar situations and came up with the similar conclusions. To make a long story short – it is very hard to justify for a business an existence of a dedicated team which does not deliver a tangible value. We completed only one project in a year which was only the part of big refactoring, although we worked pretty hard. Personally, I realized I wanted to deliver value to customers, it turned out it was a motivating aspect for me. Just re-coding what was already working well turned out to be a little fun to me. So I left the team and very soon it was dropped by the management. Most developers agreed that refactoring should be done gradually as a part of features development.


Becoming an effective remote programmer

I have been working from home as a web developer for one year now. But before that I was a web freelancer for almost 4 years and for 2 years I worked in the office at SimCorp but my whole team was sitting in Copenhagen. So, I am a pretty experienced remote worker. I love this way to work and thus I try to be effective at it. In this post, I will share what I think it takes to be an effective remote programmer (non-tooling tips, which are covered here).

I believe that in order to be an effective remote programmer, one has to deal with severe problems this way of work imposes and develop personal traits: initiative, excellent communicator, and trustworthiness.

Let’s talk about problems first.

Lack of social interaction. Or just loneliness. Working from home solved my biggest problem of working in the office – the commute. It took me 3-4 ours per day, almost 55 days per year. But unfortunately working alone is unnatural. We as species survived because we had social interaction and collaboration. We are too weak to survive alone. Thus the need to socialize and collaborate is deeply embedded in our brains.

Furthermore, researchers have proved that changeable environment and interaction with different people improve our ability to learn.

Hence, to be effective remote worker one must find the way to interact with other people outside his family. I really struggle with this. But I try my best to socialize with my current and previous coworkers, to teach courses, to talk at conferences and other events. I am in constant search for an opportunity to have my social interactions.

Some people use coworking spaces but I am not a huge fan of them since for me workspaces neglect commute absence and personalized environment benefits.

I do at least 1 hour of exercises 6 days a week. To compensate the toxic influence of loneliness on our ability to learn effectively, the remote developer must do a lot of exercises, preferably aerobic, preferably every day. Researchers proved that aerobic exercises can help with this problem.

Lack of self-motivation. It is definitely hard to stay motivated when you stay home and things don’t flow much. There is no boss standing behind your shoulder and motivating you using whatever method he is good at. In such a case I believe only hard self-discipline can keep you effective. I use Pomodoro technique and my discipline is pretty simple: deliver 10-12 Pomodoro per day to my customer. That is not less than 50 Pomodoro per week. I almost never break this rule. Sometimes I feel like I do not want to code. But I force myself into doing Pomodoros. And you know what? Right in the process I start feeling good about the work I am doing and the amount of Pomodoros I have done.

Lack of self-management. Since you don’t have 9 to 6 working day anymore your concentration starts to decay. You start mixing work with entertainment and home duties. You relax because you are home anyway. This semi-work can take a large portion of your day and at the end of the day you might feel unsatisfied with what you have achieved. Thus you merge your work and your private life and neither part of your life wins in such a situation. To stay effective, the remote worker must manage his time and his work very actively. I don’t have a manager who would micro-manage me and therefore I do everything on my own. But I still use some kind of Kanban board, create and manage tasks there, try to do estimates and reflex on my estimations accuracy. I usually pretty consistent with my working hours. Although I do have the flexibility to work whenever I want to, I don’t really wait for that muse to come and start working according to the planned schedule.

Now, let’s talk about traits one has to sharpen to be an effective remote developer.

Initiative. You don’t move your ass unless someone kicks you? Then remote work is not for you. At least for now. I hope one can teach himself to be more initiative. Initiative bases upon a strive for productivity, being self-driven and requiring the least amount of managerial overhead. From time to time, you will have the idle time. It is important to spend this time with the value for your customer. I always try to initiate some improvements in the project which will potentially increase team’s productivity in the future. These improvements might be technological, educational, process or product related. Does not matter much. The point is to strive to deliver something a value to my customer even if I was not asked to explicitly.

Excellent communicator. This is a trait everyone in tech must have. But in remote work it is critical. To be effective one must communicate as clearly as possible because almost everything happens asynchronously in remote. This constitutes a special challenge for a non-native English speaker who works for the English-speaking customer. I do a whole lot of things to improve myself on communication skills. Starting from reading books on tech writing and finishing with listening to podcasts and writing this blog.

Trustworthiness. I have never seen my current manager. He has never seen me. He sits somewhere in California and I sit here in Ukraine, opposite side of the globe. Most probably we will never see each other and have some beer together. Teambuilding is the place where people build trust in addition to other things. But we can not have them. Hence all our interactions are based on a trust in advance. He and I do our best to maintain and build real trust. I try to be never late on the call and in case something happens I make him aware. I try to make visible my work and my progress. I do my best to be as available as possible and make him never wonder where I am and what I do and whether I am worth to deal with. My manager also does something which makes me trust him. This something is only known to him. One thing I am sure about is that he know how to build a trustful relationship with the remote team.

To summarize, in order to be effective try to socialize at least once a week with people outside your family. Do everyday workouts (preferably aerobic ones), use Pomodoro technique to fight distraction and structurelessness and impose a discipline on yourself to do at least 50 Pomodoros per week. Try to be initiative, constantly invest in your communication skills and actively build the trust with your remote client or manager. There is, of course, whole tooling side of this topic, which I have covered here.


Про творчість у програмуванні

Програмування давно перестало бути творчою професією. Мені нудно ним займатися і я не хочу провести решту життя перед монітором складаючи програми. Так розмірковує один мій приятель, який між іншим, уже покинув програмування і зайнявся більш творчою справою.

Мені здається він і має і не має рацію. Професійне програмування сильно змінилося за останні кілька десятків років. Якщо відкрити та погортати книги з програмування минулого століття, можна помітити, що вони насичені інформацією алгоритмічного і математичного характеру. Думаю, це пов’язано з тим, що більшість тогочасних програмістів були колишніми або математиками або інженерами з електроніки. Апаратне забезпечення було порівняно примітивним, тому більшість програмістів займалося системним програмуванням намагаючись витиснути максимум із заліза. Було багато завдань, які ніколи раніше не стояли перед інженерами. Наприклад, створення перших браузерів, текстових процесорів, графічних редакторів та інтерфейсів. Усе це вимагало неабиякої майстерності у створенні ефективних алгоритмів і, безперечно, творчості. Робота програміста була дуже творчою, програмістами тоді були переважно фанати своєї справи, яких захоплювала можливість створювати щось, чого не існувало раніше переборюючи обмеження фізичного світу представленого примітивною електронікою.

Сьогодні усе змінилося. Так, є невеличка купка програмістів, які досі розширюють межі можливого. Вони сидять у Google, Microsoft, Nasa, Facebook та інших місцях створюючи й визначаючи наше технологічне майбутнє. Але світ заповнений величезними ордами програмістів. Більшість із них і близько не ставляться до своєї професії пристрасно. Вони потрапляють до неї лише тому що вона комфортна, безпечна і гарно оплачувана. Перед цими ордами ставлять бізнес завдання і ці завдання мають мало спільного з творчістю. Потрібно порахувати купон по облігації? Ось, будь ласка, формула. Необхідно відсортувати великий масив даних? Ось бібліотечна функція. Необхідно зробити запит до мільйонної бази даних? Ось приклад запиту на StackOverflow. Треба зручний веб інтерфейс? Хапай Angular та Bootstrap! Треба виконати автоматичну транскрипцію аудіо? Смикай IBM Watson web API. Потужне апаратне забезпечення робить економічно невигідним возіння з вишуканими алгоритмами напівосвічених програмістів з астрономічними зарплатами. Дешевше мати немотивованого програміста який пише аби як. Апаратне забезпечення і повільна мережа у 80% випадків перекриє неефективність його програм.

І все ж я вважаю себе творчою людиною, яка отримує задоволення від своєї роботи навіть після 13 років практики. Мені не потрібно застосовувати творчість для складання крутих алгоритмів. Творчість я застосовую до оформлення, до створення по справжньому простих і читабельних програм. По своїй суті це ніщо інше як педагогічна діяльність, спроба передати процедурні знання не лише комп’ютерові, а й людині, яка читатиме програму завтра. Чого вартий лише вибір назв для різних програмних артефактів! Це цікаво, це захоплює, це мотивує. До того ж простий та читабельний код має велику цінність для компаній, адже дорогі програмісти менше часу витрачають намагаючись зрозуміти код. Натомість вони мають більше часу створювати нову функціональність та виправляти дефекти.


Мій перший мобільний Веб застосунок

На 5-ому курсі університету у мене був цікавий курс, назви якого я не пам’ятаю. Щось подібне до економічної кібернетики, навіть не знаю яке він мав відношення до моєї спеціальності, телекомунікації. Я не можу пригадати жодної теми з того курсу, як і з більшості інших університетських курсів (тому я виступаю проти вітчизняної вищої освіти, для мене то марнування часу). Але дуже добре пригадую екзамен, для якого я вперше розробив мобільний веб застосунок і забезпечив гарний результат на екзамені для усієї групи.

Так, ми були не надто сумлінними та чесними студентами. Мені особисто взагалі було до всього байдуже, на той час я вже кілька років займався професійно веб програмуванням, вважав себе програмістом та збирався розвиватися у цьому напрямку. На тому екзамені було необхідно розв’язати доволі складну задачу, здається пошуку оптимального шляху для доставки товару із пункту А до пункту Б. Розв’язок необхідно було підкріпити описом ходу розрахунків, оформлених визначеним чином. Розв’язок цієї задачі найбільше впливав на результат екзамену, а задача була складнувата для більшості студентів, особливо в умовах стресу, який завжди наповнює екзамен.

Як я уже згадував, я був на той час веб програмістом і мені стало цікаво не просто розв’язати задачу, а написати програму, яка б розв’язувала задачу та автоматично формувала звіт про хід розв’язку. Посидівши деякий час я таки написав цю програму використовуючи PHP і, звісно, HTML для формування звіту про розв’язок. Програма містила форму, в яку можна було вводити умови задачі та кнопку, яка запускала обчислення.

Але як було мою програму використати на екзамені? Комп’ютерами користуватися не можна було, та й не було ні в кого ноутбуків у ті часи. А навіть якби й були, про WiFi в 2005 в університеті ніхто і не думав.

Тогочасне рішення мені винесло мозок. Це було магічно. Я почувався магом і ще більше переконався у тому що я хочу продовжувати бути програмістом. У мого приятеля був один з перших смартфонів Nokia


Він не шкодував грошей на мобільний інтернет, хотів бути онлайн 24 години на добу. Тож я домовився з ним, що він дасть мені смартфон на кілька хвилин, поки моя програма розв’язуватиме задачу. Натомість я йому дозволив скористатися своєю програмою для розв’язання його варіанту задачі. Викладач не протестував. Чолов’яга не підозрював що смартфон + інтернет міг розв’язувати його задачі.

Звісно, ми впоралися з екзаменом на відмінно і дуже швидко. Був навіть час ще декому з групи, хто погано справлявся із задачею, передати телефон. Я тріумфував, моя програма допомогла стільком людям! А я для себе відкрив новий світ мобільного інтернету.

Але до чого я це розповідаю? По-перше, мені здається ця історія цікавою, а по-друге, це чудовий приклад ефективності learning by doing. Я не пам’ятаю жодної теми з того курсу, жодного факту, жодної ідеї. Але чудово пам’ятаю цю програму і як вона працювала.