In order to use ghminer, let’s install first (you will need npm installed):

npm install -g ghminer

Then, execute:

ghminer --query "stars:2..100000 size:>=20 mirror:false template:false topic:ruby" \
  --start "2019-01-01" \
  --end "2024-05-01" \
  --tokens pats.txt \
  --json

Where stars:2..100000 size:>=20 mirror:false template:false topic:ruby is the search query to the GitHub API, 2019-01-01 is a start date to search the repositories those were created at this date, 2024-05-01 is an end to search the repositories those were created at this date. pats.txt is a file that contains a number of your GitHub PATs, separated by the line break, should look like this:

ghp_pAAAAAA......VCL7wgw
ghp_oBBBBB......XE1ySTKq

Depends on the possible captured repositories, the collection process can take a while. When it’s done, you should have both: result.csv and result.json.

That’s it, now you can use it in your researches!

PS. Here is the example of collected 14.4k Javascript repositories (took me ~57min to collect) that were created during 2023-01-01..2024-05-01.

Let’s start.

My computer is 2020 MacBook Pro 13’ with M1 chip.

Terminals I use are iTerm 2 and Warp. Configured with oh-my-zsh and powerlevel10k

My main IDE is IntelliJ IDEA Ultimate. My settings.zip.

Browser I use daily is Arc. Amazing Chrome/Safari/Vivaldi replacement.

For document writing and presentations I use LaTeX. To my knowledge it’s the only system for this purpose. Usually, I keep all my documents in Git.

For social and communications I prefer using GitHub, for chats with normal people I use Telegram, Slack and Email over HEY.

For storing passwords I choose 1Password.

iStat Menus helps me monitor my system.

Rectangle for screen management.

DeepL for translations.

As productivity suite I use these apps:

• Raycast provides hotkeys, automation and fast searching. Previously, I used Alfred 5 for this.
• Flow, for time controlling
• DevToys, just a swiss army knife for each developer

The more reasonable commits you have, the more traceable git history the project will have. That’s obvious.

But why do ween such “traceable” history of changes? To trace results right in the middle. When we have just 1 commit after year/week/day/etc, we don’t have a full “trace” of developers’ thoughts. Just 0 and 1 state. We can’t get back and take a look at some unfinished code. We can’t check the reason why this code is being changed. It’s not being developed in just one take. But our history we will have only a squashed final one.

That’s why smart people invent VCS, including SVN (2000), Mercurial (2005) and Git (2005).

By committing every reasonable change, we are doing a good service for our project. We can trace every ticket, every requirement till commits we create.

Let’s take a look at l3r8yJ/oop-cop/pull/125. It’s a pull request we managed to merge just today. This one solves this l3r8yJ/oop-cop/issues/85.

How its being done:

• Project Manager (@zoeself) assigned me on the task
• I’ve created a dirty skeleton without unit tests
• Then added a few tests
• Even more tests
• Introduced a few puzzles, and this one gets resolved according PDD methodology
• Clean it up, so Qulice says BUILD SUCCEEDED.
• @l3r8yJ approved pull request and @rultor merged it into master branch

This is a traceable development. Thanks to these Git commit messages, we can build a thought line. A reasoning. This is an incremental contribution.

The result is:

• Each person can take a look at how this ticket being resolved
• Puzzles being converted into new tickets

So, the more traceable your commits, the more maintainable and transparent is the project. Many incremental commits a day. That’s the way.

Check a few more examples: eo-cqrs/eo-kafka/pull/429:

• @l3r8yJ reported a bug
• @l3r8yJ created a few smart commits
• @h1alexbel (me) asks about tests
• @l3r8yJ introduced new puzzles, in order to cut corners
• @h1alexbel (me) checked again, make a comment
• @l3r8yJ responds me back, we merged

objectionary/eo/pull/2731:

• I found an old issue
• Created a few commits including skeleton, adding/removing object from Place.java
• Reported a broken test with @Disabled + puzzle, as explained here
• @maxonfjvipon asked me to resolve code duplication inside Place.java that task also mentioned
• I fixed that, requested a re-review
• @maxonfjvipon approved
• @yegor256 (an Architect) approved too and @rultor merged

Problem

Nowadays, all modern Frontend frameworks are completely displaced HTML templates we had in the MVC era.

What have we got?

• Components
• Handlers
• Services and other great code bloaters.

All of them are help only to violate Thin Client Pattern by adding extra complexity on plain HTML most of us need it these days.

By keeping all the code inside one simple monolith, we can boost productivity and reduce the time required to deliver full-stack features to the end-users.

The main problem is that this extra complexity is not required. For most of us. And we want to reduce delivery time by utilizing the power of a team that consists just of a few programmers and designers. The team where each programmer can deliver full-stack features. Right from the backend to the Frontend part.

What’s Wrong With MVC?

No SPAs/MPAs, lets get back to MVC! Our backend now will be responsible for aggregating objects and placing them into HTML Templates and rendering nice HTML pages. Plain Old MVC!

Don’t rush it.

MVC is not the best here, I believe. Well, HTML Templating is a good idea, it will improve your app from simplicity side, and so, but most all modern Web Frameworks will force you to not have an API in that case. Instead, you will have a Controllers that return you HTML templates, you will manage your model to place all its fields to that template.

The result of that is that other apps cannot integrate you, only crawl, maybe.

Ideally, application should expose both interfaces: Web pages to the end-users, a.k.a User Interface, and an API, objects that your application stores/handles/sources. That’s the separation of concerns. That’s the best scenario.

What options do we have for this one? For now, I think we have only one option that will do this: XML+XSLT+HTML.

And that one is not ideal either. A way more complex, and there is no JSON have certain limitations.

Conceptual Solution

Imagine you have an Application, written in Java, that needs to be able to render users. First, let’s define a RESTful endpoint that will return us JSON response:

GET /api/v1/users

And write some code for our “controller” (it is a conceptual code from @l3r8yJ/towers):

new ApiRouteOf("/api/v1/users", new JsonUsers(...)).serve();

Running our app, this endpoint should respond to us with JSON like this:

[
  {
    "email": "test1@test.com",
    "id": "4b53a171-139c-4808-b61c-fcad879b5084"
  },
  {
   "email": "test2@test.com",
   "id": "a3303252-343c-4c4f-baf1-f4f5a6f333a4"
   }
]

now we have a RESTful endpoint serves User Resources as JSON, that we can integrate, process it somehow else, and so on.

And now we need some View/User Interface for this data. We can do this by “wrapping” the API endpoint and placing Templates on top of it. Something like this:

new Templated(
  "/users.html",
   new ApiRouteOf(
     "/api/v1/users",
      new JsonUsers(...),
      new Render("users")
    )
).serve();

Finally, we create an HTML Template users.html:

<% users >
...

Thus, our App can serve both API to integrate and HTML View to the end-users.

This is a concept on which we are working right now. What do you think? Don’t hesitate to submit your ideas in the comments and here.

Traceability and Clearness

The main problem is that we’re trying to solve is that most projects are untraceable and unclear enough for their crew.

• “Why is this decision made?”
• “Why I get less than John doing the same Java coding?”
• “How to check out the code before new feature requests are employed?”

We are here for clearness and traceability on your project.

Documents Under Control

In 2023, most software teams manage quality gates, a.k.a CI/CD pipelines in their code. Why can’t we apply it to the documents we have? Project as a Code means that. Each change is controlled since kept in git and has to be pass a merge pipeline before getting into master branch.

Thus, all the software documents including requirements document, Use-Cases, diagrams, and others can be defined in the source code and empowered with control and integrity checks, or even with AI post-analysis tools.

Tickets already can be defined in the code, why documents can’t be?

Here are a few ideas of how it can be defined:

The project definition:

project:
  name: ai-secret
  description: AI Secret Project
  budget: 1mil$
  private: true
  date:
    start: 2023-10-03
    end: 2023-11-30
team:
  - email: aliaksei.bialiauski@hey.com
    roles: ARC
  - email: john@google.com
  - roles: DEV
  - email: mary@gmail.com
  - roles: [PM, PO]
stakeholders: ...

Use-Case definition example:

name: where User can identify himself
rdql: |
  The user logins with his credentials, if he has them.
  Otherwise, he registers in the system first.

Results are Traceable

Since we have all the stuff in the source code, we can check, integrate and validate artifacts and results. Thus, productive work can end only with lines of code, or new documents but not with new Slack messages or meetings.

Also, a project means a role-based project. We don’t know how to make customers happy. Developers know how to develop Java classes; PMs know how to manage a project.

Since this, a properly managed project’s required not just people, or just programmers. We need concrete individuals capable in their realms. The Goal is to find the best and segregate them using simple project roles.

Happy or Quit

All the team members know everything about the project. They know how to boost their salary, what kind of performance project expects from them, what are the project risks, and so. They know what are the rewards. They know how they will suffer if a task is not completed or deadline missed. Simply put, they are not slaves kept in the dark.

On the project you are happy and productive, or you quit.

No Backlog, Puzzles instead

Project as a Code inherits Puzzle Driven Development. All the bugs kept as puzzles. Puzzles are defined in source code and discovered while working on tasks. Tasks mean new tasks down the road. But instead of keeping all of them in our minds, we are simply doing it in the code.

Thus, we are slightly evicting a PM role from the definition of done. Now, we are formulating by us, people who know the reality.

Ecosystem

Moreover, this philosophy can be employed with a new software development platform for enabling that kind of productivity for all team members and stakeholders on a project.

We can go even further and make the platform programming language native. I don’t think it can be Java/Kotlin or TypeScript. Simply because those langs are good in enterprise/product development, but we need some sort of DSL, like Ruby and Groovy, or just plain XML/YAML sheets.

Open source is a must. But open source is not about “free” software development. At all.

I think it’s about quality and reusability. First you develop on demand: pull requests, new bugs by strangers from whole the world arrived. You merge them and a new version of the product gets released.

How to make money on product in open area? Let me explain my thoughts on this.

I think the best way to do it is to leave your product “free” and open-source and sell services and service management on top of your product. HashiCorp is doing the same what I described, maintaining Terraform and selling Terraform Cloud or whatever.

While other companies like Spacelift, offer the same services as Terraform Cloud. In terms of business, HashiCorp doing everything OK by restricting license. So, for instance, Spacelift will be required to pay for Terraform usage after Terraform 1.6.

But along with that, Terraform and other HashiCorp open source products will lack off quality and enthusiasm in their repos. I’m not interested in contributing to the product that I can’t use if one day I decide to create some service that uses Terraform.

It’s sad.

Presenting is a skill that you need to actively master as a software developer. But when you’re presenting your ideas, keep your point very specific and please just don’t throw all the facts you want to say.

It’s better to say less, but more specific, strictly connected to one point. Create a general idea for your keynote and inject it going through the whole speech.

Here are the areas where it can be applicable:

Architecting

Instead of showing the full picture, create a simple design pieces and deliver them continuously. Your team will learn and puzzle faster, simply because design sketches will be specific and granular for each task. Instead of showing how smart you are, focus on a problem.

Teaching

When it comes to mentoring somebody, I’m also making this mistake quite often. Day to day you need to deliver a little bit of new information. Otherwise, your students will be overwhelmed of disconnected in their mind facts.

Conference Talks

Being in the conference talk, you need to be specific on one thing as only you can. Thus, people can better understand your ideas, instead of just shrug and walk away.

If you are a senior developer, you have to know how to deliver your ideas to many people in front of you.

I strongly believe that being a developer, besides coding, you need to know how to help your project. Remember, when coders create lines of code, professional developers create software that works.
What are the steps to build from zero some more or less stable software product?

According to RUP, there are 9 disciplines in a common software project:

• Business Modeling
• Requirements
• Analysis & Design
• Implementation
• Testing
• Deployment
• Configuration & Change Management
• Project Management
• Environment

All of them you need to be aware of!

And don’t get me wrong. It’s not about being Jack of all Trades. You need to be a niche specialist, a niche-focused expert. Imagine X & Y coords, on X is your disciplines, while on Y is your tech stack you’re expert in. If you are a niche-focused expert, and at the same time, know all activities in software project, you will the MVP on each project.

I’ve summarized all the RUP phases into the following disciplines, the developer realms:

• Business Modeling
• Requirements
• Design
• Implementation
• Testing
• Deployment
• Configuration
• Project Management

Business Modeling

First things first. Business modeling. Here you need to have knowledge of use-cases modeling and all things related to inception phase. Try to follow simple rule: docs over talks.

Requirements

Then, after business modeling, we need to start collecting requirements. To do it right, you need to know what SRS is. Moreover, you need to be aware of rules and best practices when it comes to requirements. And again, start preparing the right documents. I’m usually creating one long README.md file and then commit it into git repo. So, your tooling can be: Git + Markdown/LaTeX.

Design

To become an architect, you definitely need to become very successful here. First, master your UML skills. You must be able to draw simple UML sketch, that will be understood by all stakeholders, not only by your developers. So, simplicity is also your instrument. I believe that the simpler you present and sketch, the better architect you are.

You need to know how to formulate design not only in diagrams, but also in text. Learn SAD by RUP or SDD by IEEE. I suggest you to pick one standard and integrate it in your README. Again, produce documents that emphasize all your decisions.

Implementation

Finally, coding! Your main focus should be on Software Maintainability. It’s lots to learn how to reach it. It’s hard. Your open source skills can boost it significantly. You can gain this skill only by creating and maintaining your own software packages and products. Simply put, code reusability is the main virtue here. While duplication is a sin.

Testing

First, understand why we need testing. It’s not about comparing actual and expected results. It’s about finding bugs in your software.

As software developers, we need to know how to create maintainable tests. Here, maintainability reflects on your design. If you have a lot of procedural stuff in your production code, you will notice that it’s so hard to maintain tests for such code. I even believe that to become a better developer, you need to focus mainly on automated tests and test architecture. Some tools that will help you here: unit, integration, e2e, load, performance testing. Code coverage and any test control are also applicable here.

Deployment

My favorite here. You need to be well-versed how your software gets deployed to the end users. No excuses. Popular tooling is some cloud provider like AWS or similar, Kubernetes, and Docker. And your deployment must be configured at the start of the project and be fully automated like here.

Configuration

You need to know how to configure your CI/CD pipelines, how to set up VCS.

Here, you can also merge Deployment activities and configuration management by using IaC. Terraform is a good one.

Project Management

To grow, you need to understand project management. I’m not talking about Agile fairy tails, I’m talking about some serious project management rules and best practices. And you can’t learn them in one day. From myself, I suggest to read PMBOK, CMMI and try to apply them into your project workflow.

Let me summarize all the stuff: you need to go full cycle. The best way to do it is to create your own pet project. There, you will practice all the disciplines, and you will be able to do them in a large-scale project.

First things first. Imperative programming covers 99% of code. We’re still writing the same procedural code, that we did in C 50 years ago. We are done with it. It’s time to cure our brains.

Scalars are Declarative

Scalar Driven Development is a way to model our objects as declarative parts, some results, that will be computed later.

We define the interface first, where we denote our Scalar object.

Here is an example:

interface User extends Scalar<Json>{
  @Override
  Json value();
}

Our Scalar from User is a JSON document. By the way, Scalar<T> taken from cactoos library.

Then we are creating a few implementations for it:

final class Simple implements User {
  
  private final String username;
  
  @Override
  public Json value() {
    return new MutableJson().with("urn", this.username);
  }
}

No big deal, just giving username and building a JSON document from it. Someday, the logic will scale or change completely. But we want our object to stay the same, while adding new features. The Decorator will be in charge:

final class Validated implements User {
  
  private final Scalar<Json> user;
  
  @Override
  public Json value() {
    // validate json
    return this.user.value();
  }
}

Now we can validate our username before applying it to JSON document storage. Frankly speaking, it’s not a plain decorator, we are chaining a scalar by passing it.

final class Flushed implements User {

  private final Scalar<Json> user;
  private final Documents docs;

  @Override
  public Json value() {
    this.docs.apply(this.user.value());
  }
}

That’s a composition of objects, where each of them is a good citizen of a pipeline. Each of them is represented as a small result, as a part of a bigger one.

Delayed calls

Nothing will be computed until value() is called:

new Flushed(
  new Validated(
    new Simple(
      "Aliaksei"
    )
  ),
  docs  
).value(); // trigger pipeline

Scalars vs. Workers

XmlParser, BookMapper, UserValidator. Who are they?! I think they are great candidates for refactoring…

Scalars also will help you to avoid evil suffix of your objects: -ER, a.k.a Worker, we just need to define our results of manipulations with objects as other objects, that’s how Scalar is born.

“Declarative programming means you define results of manipulations with objects as other objects”.

Let’s see an example: we need to map HttpRequest to Song object:

interface Mapper<F, T> {
  T to(F from);
}

final class SongMapper implements Mapper<HttpRequest, Song> {
  @Override
  public Song to(final HttpRequest rq) {
    return new Song(rq.path("name"));
  }
}

// call it
final SongMapper mapper = new SongMapper();
final Song song = mapper.to(request);

SongMapper encapsulates nothing, frankly speaking it’s not even a class, it’s nothing more than a set of procedures. More than that, SongMapper represent nothing from a real-world. We are just treating it like some smart object to manage our dumb DTOs.

Instead of doing such procedural things, we can go with SDD; it will make our objects more declarative:

final class SongFromHttp implements Scalar<Song> {
  
  private final HttpRequest request;
  
  @Override
  public Song value() {
    return new Song(this.request.path("name"));
  }
}

// call it
final Song song = new SongFromHttp(request).value();

The same functionality, but now:

1. we are treating SongFromHttpas real-world entity.
2. SongFromHttp has a state inside self: object encapsulates HttpRequest
3. thanks to 2., we can perform lazy computation of SongFromHttp#value()

Watch this:

Summary

What are we learning here? Here is my summary of what SDD means:

1. State (your object must encapsulate something)
2. Declarative (scalars are treated as parts of other objects, as some results)
3. Delayed calls (unless value() is called, nothing will happen)
4. Absence of getters (there is value(), no sense to violate encapsulation)

There are a few real-world examples of how we maintain those ‘scalars’ in our projects: eo-kafka, cmig, cactoos and many more at EO-CQRS.

That’s it.

P.S
@l3r8yJ called this way of development - Scalar Driven Development (SDD).

Now, in 2023, modern programming languages are focusing only on one thing:
How to satisfy all the developers in the world, that they can use only our language to solve all their problems?

The Biggest problem we have now, that the languages we’re dealing with are the products of the majority votes. Do we need reflection and annotations? Most developers will say “yes”, no matter how bad is reflection. The same with static methods. The same with 90% of features in “modern” programming languages such as Java, Kotlin, TypeScript, Go, Ruby, etc.

While languages and platforms formulate the maintainability of the programs we create, modern languages start looking uglier, uglier, and uglier. To satisfy all the needs, modern languages inevitably start introduce ability to write the same things in a different way, a.k.a flexibility, or syntactic sugar.

Simply put, higher quality comes from simpler languages. - Yegor Bugayenko

Just look at JEP 445: Unnamed Classes and Instance Main Methods, the Java 21 preview feature:

Java main method and class a.k.a entry point can be presented now in 2 ways:

public class Entry {
  public static void main(String[] args) {
    // logic
  }
}

void main() {
  // logic
}

The same happens with all the languages. They introduce new and new features to satisfy all the developers around the world and boost their creators on the stock market (Google, Microsoft).

What is the solution?

I believe that now, we need more high-level languages with focus on specific use-cases. It should be simple, high-level, and specific-focused. That’s it.

There are some features of really new and modern programming language, based on D29:

1. VSC support at language level
2. UML and OCL support at language level
3. CI/CD support at language level
4. Declarative programming
5. Distributed objects/functions and their scalability
6. Cloud Computing
7. Data formats e.g., XML, JSON, YAML, CSV
8. Testing: unit, integration, end-to-end, performance
9. Multithreading
10. Documentation
11. Software licenses
12. Low-code compatibility
13. Runtime platforms e.g., Docker, K8s
14. Object Metadata
15. Object metrics
16. Component repository: e.g., Sonatype Nexus
17. Design patterns indication
18. Compilation into other programming languages
19. No scalar types e.g., int, boolean
20. No data flow control e.g., for, while
21. No reflection