Web Development: Overview & Basic Concepts
· 10 minute read
Last updated: May 7, 2022
· Published: Sep 20, 2020
You want to learn web development? Then let me give you an overview & some pointers on some important basic concepts:
When non-technical people are thinking of the web, they are thinking of web sites that they can access via their browser. That is why we will focus on that first. The most important concept to understand here is “request and response” aka “client and server”. This is how a website works:
- Web browser sends a request to a URL like “http://www.markusdosch.com/"
- Behind “markusdosch.com”, there sits a server (a fancy name for a computer that “serves” stuff). The server receives the request and generates a response. Very often, this response is just some text, such as “Hello World”.
- The browser receives the response.
- The browser knows how to display the response to the user.
In case of simple text, the browser will simply display this text as-is. More often, the text looks like this: “
<h1>Hello World</h1>”. This is HTML code. If the server tells the browser that its response has HTML as content type, the browser will “render” the document instead of simply putting out the raw response text. This is how every website works - the browser interpreting the response and rendering it as a nice document. The browser already has some default styles built-in. But most often, the website developer will want to tell the browser how to exactly render the elements, e.g. using which colors, or which element sizes. The usual way to do it is via CSS.
Using a special type of tag, the “anchor” tag ("
<a>”), the browser can render links to other URLs. When a users clicks a link, a new request is initiated, the server generates a new response (a different response, as the URL is different now), and a new website is rendered by the browser. That is the whole magic behind basic websites - request and reponse.
In this basic model, we can make a separation between two disciplines:
- Web design & frontend development: This discipline is concerned with creating good-looking HTML & CSS layouts, that present a website’s content in a good way.
- Backend development: This discipline is concerned with developing the program that runs on the server and that decides what to do when a request is received (mainly: What kind of response to send back).
Static files vs. backend development
In the simplest way, you don’t need much backend development. Instead, you could simply have some static HTML files on your server and respond with these. E.g., when a request “/” is received, you could answer with a file
index.html, and when the request is
/blub/hello.png, the server would answer with the file on path
/blub/hello.png. For simple websites that just need to display some static content, this is actually enough. It is quite easy to configure your server to act as a simple “static file server”. This website, for example, is such a static website. But instead of writing all HTML files by hand, I generate them using the static site generator Hugo.
Backend development comes into play when you want your responses to be dynamic, i.e. to potentially be different for every request. E.g. you might want to include the current time in the response, or save or display some data from a database. And this is were things become really interesting on the backend side, and an area full of possibilities emerges…
You can do backend development in any programming language. The only requirement is that your program follows the HTTP protocol, because that is how browsers send their requests and expect the responses. Either a language has already built-in support for the protocol, or the language-community provides packages that make interacting with the protocol easy.
What is the HTTP protocol about? Basically, it specifies a set of methods for client-server communication, with the most important ones being
POST. A client, e.g. the web browser, initiates a request to the server and specifies which request it is following. As part of the request, some data is sent to the server. The most important being the URL. Less user-visibile data includes headers (e.g. with cookies), but this is out of scope for this document. A
GET request is basically just sending a URL to the server. The URL can include parameters, the so-called “query parameters”. Example:
http://markusdosch.com/blub?language=de_DE&name=Tom sends two query-parameters to the server. The server can read these parameters and adjust its response according to the parameters (e.g. greet the user with the
name parameter). A
POST request can additionally contain a request body. The request body can be of any type - in the simplest form, the body’s content takes the same form as the URL query string, e.g.
Sending data via the request body has some advantages over sending it over the URL. One is that it allows much more content to be sent (a URL has a certain character limitation). Also, as the body can take any form, you could also send data in other formats, e.g. JSON or XML. This might not seem important now, but becomes important once AJAX comes into play later. When the client sends a body, it has to specify the body’s content type. In the case of the “URL query string equivalent”, the content type is called
Why is it “
Here is an example HTML form that a browser would be able to render:
What does it do? When a user fills in the form data and clicks “submit”, a
GET request will be issued to
http://markusdosch.com/submit?name=<NAME_INPUT_VALUE>&age=<AGE_INPUT_VALUE>. If the form had set
POST request to
http://markusdosch.com/submit with body
name=<NAME_INPUT_VALUE>&age=<AGE_INPUT_VALUE> (and content type
application/x-www-form-urlencoded). In any case, the server could take this request, read the request parameters, react to them, and return a response. There are certain conventions on what type of effect what HTTP request should have (e.g. that
GET should not change or create anything user-visible on the server. So subsequent
GET requests always return the same result), but this is out of scope here.
Just knowing about HTML forms & backend programming can take you quite far with developing a website. You could e.g. now implement a rudimentary version of Twitter, Facebook, Wordpress etc. On a conceptual level, you now have all the tools to create such CRUD (create-read-update-delete) applications. Of course, these businesses are infinitely more complex and have much more logic going on, but still, on a conceptual level, they are not that hard.
Requests from browser, some business logic on the backend, resulting in a response that gets rendered by the browser - this could all be quite easy. But has a big downside: For every user interaction, a complete fresh page load has to be done. This limits the scope of applications that can be realized. Some simple patterns that would not be possible: Endless scroll in Facebook, submitting a comment while watching YouTube, telling you a username is taken while you are typing. Some large patterns that would not be possible or look very different from what we are used to today: Applications with high interactivity like Google Docs, multiplayer browser games, chats…
This is why client-side programming is a thing. Over the years, browsers have changed from simply rendering static responses, to allowing a full programming language commands to be executed within the browser context. This is very important - this is actual code & logic being executed on the user’s own machine, instead of on the server! This is huge!
- AJAX: Sending requests to a server without triggering a page reload (and handling the response).
Frontend-heavy websites (and API-only backends)
The leaves us with a clear separation between an
app (frontend) part, and an
If your website is actually more of a “desktop-app gone web” (think Word, PowerPoint, Photoshop,…), then you should look into the frontend-heavy direction. You have much more options for interactivity there, but as best practices are still emerging, you likely have to make more choices on your own.
I think, a lot of the tools you are reading about in the internet (e.g. in guides, blog posts, or discussions) become much clearer and easier to learn once you have the big picture of web development in mind. I hope to have done my part in showing you the “big picture”.
I hope the web is clearer for you now! 🤓🕸
If you have any questions or ideas for follow-up paragraphs or articles, don’t hesitate to message me (email@example.com)!
- Customizing my shell: From bash to zsh to fish
- JSON5: JSON with comments (and more!)
- jq: Analyzing JSON data on the command line
- Get Total Directory Size via Command Line
- Changing DNS server for Huawei LTE router
- Notes on Job & Career Development
- Adding full-text search to a static site (= no backend needed)
- Generating a random string on Linux & macOS
- Caddy web server: Why use it? How to use it?
- Tailwind CSS: A Primer