diff --git a/docs/guides/applications/big-data/machine-learning-cyber-attacks/index.md b/docs/guides/applications/big-data/machine-learning-cyber-attacks/index.md deleted file mode 100644 index 9403a307842..00000000000 --- a/docs/guides/applications/big-data/machine-learning-cyber-attacks/index.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -slug: machine-learning-cyber-attacks -title: "The Most Common Cyber Attacks on Machine Learning Applications" -title_meta: "Common Cyber Attacks on Machine Learning Applications" -description: 'Common machine learning cyber attacks include evasion, poisoning, and inference attacks. In this guide, learn about each attack and the areas of an ML application they target.' -authors: ["John Mueller"] -contributors: ["John Mueller"] -published: 2022-05-20 -keywords: ['machine learning cyber attacks','evasion attacks against machine learning'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -[Machine learning (ML)](/cloud/guides/history-of-machine-learning) algorithms and models ingest large amounts of data and use pattern recognition to make predictions and adjustments based on that data. ML powers chatbots, product recommendation systems, self-driving cars, and assists in decision-making in the health and financial sectors. Due to the prevalence of tools and frameworks like [TensorFlow](/cloud/guides/how-to-install-tensorflow) and [PyTorch](/cloud/guides/pytorch-installation-ubuntu-2004), developers are now able to add ML to their applications with less effort. Before getting started with machine learning, you should be aware of the most common machine learning cyber attacks. When thinking about the security of your ML application, you must consider the following areas: - -- **Data**: If your data is corrupted in any way, you will not obtain reliable our useful results from your machine learning models. -- **Application**: When a model becomes corrupted, even the most perfect data produces incorrect results. -- **Output**: An application only produces the output it’s designed to provide. Altering an application to perform tasks other than what it was designed to perform is a misuse of the application. -- **User**: Even if all of the other security factors for a machine learning application are correct, users can easily thwart machine learning applications through various means, such as purposely providing bad input or misinterpreting the output. - -This guide discusses the top security vulnerabilities that you may encounter in a machine learning project. Some of these vulnerabilities are common to all software development projects, while others are unique to machine learning ones. - -## Evasion - -The evasion attack is the most common issue facing machine learning applications. This attack seeks to modify input data in order to "trick" ML classifiers. For example, a successful evasion attack can insert a malicious algorithm into your application that slightly modifies an image, causing it to be misclassified by the machine learning algorithm. An evasion attack seeks to infiltrate a system in the following ways: - -- **Attachment**: An attachment can contain malicious code that executes the moment the file is opened. -- **Link**: The malicious code executes as soon as the resource pointed to by the link is opened. -- **Image**: Viewing an image within a user’s email setup can invoke the malicious code. -- **Spoofing**: A hacker impersonates a trusted party. -- **Biometric**: Using specially crafted code or other techniques, the attacker simulates a facial expression or fingerprint to gain access to a system. -- **Specially crafted code**: It’s possible to train a machine learning model to perturb the output of a target model. - -## Poisoning - -A poisoning attack is orchestrated by injecting false information into an application’s data stream, with the goal of producing inaccurate results. There are a number of situations where poisoning may occur. Here are the most common: - -- Using bad data during model training from unreliable or unvetted sources. -- Providing large amounts of skewed or biased input after model training. - -The attacker usually prefers stealth in this case because the goal is not to bring the system down. Instead, the attacker seeks to change the output in a manner that favors the attacker in some way. [SVM classifiers](https://secml.readthedocs.io/en/stable/tutorials/05-Poisoning.html) are often the focus of such attacks because the attacker uses them to perform tasks such as to redraw political or sales boundaries, or to give a particular product an edge during a sales campaign. - -## Inference - -If a hacker determines which records from a dataset are used to train a machine learning model, that information can be used to look for vulnerabilities. An inference attack uses data mining and analysis techniques to gain knowledge about the underlying dataset. In most cases, the best results come from [overfitted models](https://en.wikipedia.org/wiki/Overfitting). Overfitting happens when a machine learning model follows the original data points too carefully. This makes it possible for the hacker to query a particular data point with relative ease. This attack vector currently works only on [supervised learning models](/cloud/guides/introduction-to-machine-learning-training-and-inference#an-introduction-to-training-and-inference) and [Generative Adversarial Networks (GANs)](https://en.wikipedia.org/wiki/Generative_adversarial_network). - -As a hacker sends queries to the model, the model makes predictions based on the confidence levels for each class that the model supports, giving the hacker valuable insights into the underlying application. The worst part of this particular attack is that [it’s often used against specific people and their data](https://medium.com/disaitek/demystifying-the-membership-inference-attack-e33e510a0c39), such as their medical records. - -## Trojans - -A trojan employs various techniques to create code or data that looks legitimate, but is really designed to take over the application or manipulate specific components of it. The concept of using a trojan against an application is old and is used against applications of all stripes. However, in the case of machine learning, the trojan often remains hidden and seeks to discover more about the data used by the machine learning application, rather than performing more overt tasks, such as deleting files. There are many kinds of trojan attacks, but the list below contains some of the most common for machine learning: - -- **Backdoor**: Creates a backdoor on the target computer that the hacker can use to remotely control the computer. Remote access allows the hacker to do just about anything desired, including downloading your dataset or model, corrupting a dataset, or causing the model to perform in an unexpected way. -- **Banker**: Focuses on a strategy for obtaining or manipulating financial information. When considering the machine learning aspect of this Trojan, you must think about the sorts of information that this Trojan could obtain, such as membership inference, to obtain data, or evasion, to potentially obtain credentials. However, the goals are always to somehow convince a user to download a payload. -- **Downloader**: Targets systems that are already compromised and uses its functionality to download additional malware. This malware could be anything, so look for any sort of unusual activity that comprises any part of your system, including your data. -- **Neural**: Embeds malicious data into the dataset to create a condition where an action occurs based on an event, like a trigger. In most cases, the attack focuses on changing a neural network's weights to apply to only certain nodes. This kind of Trojan is most effective against Convolutional Neural Networks (CNNs), although current research shows that you can also use it against Long-Short-Term-Memory (LSTM) and Recurrent Neural Networks (RNNs). - -## Backdoors - -This kind of attack uses system, application, or data stream vulnerabilities to gain access to the underlying system or application without providing required security credentials. The focus is on the neural network itself, rather than on specially prepared inputs. Even though this attack is data based, the attack focuses on corrupting the neural network, as is the case with a trojan. The backdoor attack relies on an attacker modifying training data in some manner to gain access to the model, usually through the underlying neural network. Because this kind of attack is so subtle, a separate application is often required to locate and get rid of it. - -## Espionage - -An espionage attack involves stealing classified and sensitive data, or intellectual property to gain an advantage over a person, group, or organization. Essentially, this kind of attack involves stealth to [spy on an organization’s activities to obtain a particular result](https://wwwfr.uni.lu/snt/news_events/new_machine_learning_methods_prevent_cyber_espionage). The attack can go on for years because the attacker’s goal is to remain undetected for as long as possible. The results of the attack are normally subtle too, such as redirecting some, but not all, sales to a particular product. This form of attack can target machine learning data and models. It locates the data by using predictive models to look through logs for particular patterns of access. - -## Sabotage - -Sabotage performs deliberate and malicious actions to disrupt normal processes, so that even if the data isn’t corrupted, biased, or damaged in some way, the underlying processes don’t interact with it correctly. Often, sabotage is highly detectable, but at the moment of detection, it’s already too late to do anything about it. [Financial institutions are particularly susceptible to sabotage](https://gizmodo.com/banks-using-ai-are-ripe-for-russian-sabotage-report-1848687118) due to the incredible amounts of data used to create and manage models. In addition, sabotage is often hard to fix because the underlying data must be remediated first and then the model rebuilt. - -## Fraud - -Fraud occurs when hackers rely on various techniques, such as phishing or communications from unknown sources, to undermine system, application, or data security in a secretive manner. This level of access can allow for unauthorized or unpaid use of the application and influence ways in which the results are used, such as providing false election projections. Fortunately, there is a lot of research pending that also uses [machine learning techniques to detect and help mitigate fraud](https://spd.group/machine-learning/fraud-detection-with-machine-learning/). - -## Conclusion - -Before adding ML to your project, you should know about the types of cyber attacks that are frequently targeted at machine learning powered applications. Evasion, poisoning, and inference are some of the most common attacks targeted at ML applications. Trojans, backdoors, and espionage are used to attack all types of applications, but they are used in specialized ways against machine learning. Now that you are familiar with the cyber attacks to look out for, you can get started creating an ML powered application, by [installing TensorFlow on Ubuntu 20.04](/cloud/guides/how-to-install-tensorflow). - - diff --git a/docs/guides/applications/cloud-storage/access-your-box-account-from-your-linode/index.md b/docs/guides/applications/cloud-storage/access-your-box-account-from-your-linode/index.md deleted file mode 100644 index ead127a7d4e..00000000000 --- a/docs/guides/applications/cloud-storage/access-your-box-account-from-your-linode/index.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -slug: access-your-box-account-from-your-linode -title: 'Access Your Box.com Account from Your Linode' -description: 'Box is a popular cloud storage and file sharing service. This article will show you how to access your Box account from your Linode using WebDAV.' -authors: ["Tyler Nelson"] -contributors: ["Tyler Nelson"] -published: 2015-11-06 -keywords: ["box", "box.com", "cloud", "cloud storage", "file storage", "file", "webdav", "davfs", "davfs2"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- - -If you've discovered [Box](https://www.box.com/) then you know that it can be a great tool for storage, moving and managing files. The following tutorial helps you install and configure a free piece of software that facilitates Box access from your Linode. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -{{< note >}} -This guide requires having a Box account. -{{< /note >}} - -## Set Box's Mount Point - -The following step will create an empty directory where Box will live and all of your Box files and folders will appear. You can mount it anywhere, but `/home/example_user/box` will be used for this guide. - -1. Create a mount point: - - mkdir ~/box - - {{< note respectIndent=false >}} -If only your `example_user` needs access to the Box account contents, making the mount point in that user's `/home` directory will be fine. If multiple system users (other than root) need access to the Box account, then the mount point should be placed in a system directory such as `/mnt/box`. For more info, see [the davfs man page](http://linux.die.net/man/8/mount.davfs). -{{< /note >}} - -2. Add Box to fstab. - - The fstab (or file systems table) file is a system configuration file commonly found at `/etc/fstab`. It contains the necessary information to automate the process of mounting. Add an entry for your Box account: - - {{< file "/etc/fstab" >}} -https://dav.box.com/dav /home/example_user/box davfs rw,user,noauto 0 0 -{{< /file >}} - - -## Configure WebDAV and User Permissions - -1. Install davfs2, the WebDAV backend which is used to communicate between your Linode and Box account: - - **CentOS** - - sudo yum install davfs2 - - - **Debian / Ubuntu** - - sudo apt-get install davfs2 - - When asked if unprivileged users should be allowed to mount WebDAV resources, choose `Yes`. - - **Fedora** - - sudo dnf install davfs2 - -2. Give your user permission to mount using davfs2. Replace `example_user` with your user name. - - sudo usermod -aG davfs2 "example_user" - -3. Reboot your distro. This is the best way to be sure there are no user sessions lingering open. If there are, you'll experience problems mounting the Box drive even after adding your user to the proper group. - - sudo reboot - -4. SSH back into your Linode. - -5. The WebDAV share exported by Box.com does not support file locks. Thus, you need to disable file locks in the davfs2 configuration file. Otherwise, you will encounter "Input/output error" while attempting to create a file. - - echo 'use_locks 0' >> ~/.davfs2/davfs2.conf - -6. Add your Box account info to WebDAV's secrets file, replacing both `email` with the email address you use to log in to your Box account and `password` with your Box password. - - echo 'https://dav.box.com/dav email password' >> ~/.davfs2/secrets - - {{< note respectIndent=false >}} -If your password contains quotation characters (`'` or `"`), you'll need to edit the secrets file directly in a text editor. -{{< /note >}} - -7. Make the `secrets` file readable to only its owner: - - chmod 600 ~/.davfs2/secrets - -## Mounting and Unmounting Your Box Drive - -1. To mount and change into its directory: - - mount ~/box - -2. To unmount: - - umount ~/box - -## Wrapping Up - -To confirm that your Box drive is mounted: - - df - -The output should look similar to this: - -{{< output >}} -Filesystem 1K-blocks Used Available Use% Mounted on -/dev/root 4122048 886316 3009636 23% / -devtmpfs 505636 0 505636 0% /dev -tmpfs 507504 0 507504 0% /dev/shm -tmpfs 507504 1420 506084 1% /run -tmpfs 507504 0 507504 0% /sys/fs/cgroup -tmpfs 507504 0 507504 0% /tmp -tmpfs 101504 0 101504 0% /run/user/1000 -https://dav.box.com/dav 10485756 72 10485684 1% /home/example_user/box -{{< /output >}} - -To see the mount options with which your Box drive is mounted: - - cat /proc/mounts | grep box - -The output should show the following: - - https://dav.box.com/dav /home/example_user/box fuse rw,nosuid,nodev,noexec,relatime,user_id=1000,group_id=1000,allow_other,max_read=16384 0 0 - -You're done! The directory `~/box` will now reflect your Box contents! The first time you access the folder it may take a few minutes for the contents to synchronize. After that, folder access is almost immediate. diff --git a/docs/guides/applications/cloud-storage/dropbox/index.md b/docs/guides/applications/cloud-storage/dropbox/index.md deleted file mode 100644 index 1cd34d2ad35..00000000000 --- a/docs/guides/applications/cloud-storage/dropbox/index.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -slug: dropbox -title: 'Installing and Configuring Dropbox' -description: 'Installing and configuration Dropbox on a Linode' -authors: ["Linode"] -contributors: ["Linode"] -published: 2014-04-30 -modified: 2015-06-29 -keywords: ["Dropbox", "debian", "centos", "fedora", "ubuntu", "headless", "storage", "cloud storage"] -tags: ["ubuntu", "debian", "centos", "fedora"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- - -![Installing and Configuring Dropbox](installing-and-configuring-dropbox.png "Installing and Configuring Dropbox") - -Dropbox allows for the storage of your documents, files, videos, and photographs. Whatever you choose to store will be available on the Dropbox website, as well as any computers, phones, or servers you have the Dropbox application installed. - -Prior to setting up Dropbox on your Linode it is recommended to follow the [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide. You will need a [Dropbox account](https://www.dropbox.com/). Dropbox can be used on Debian, Ubuntu, and any Red Hat Enterprise Linux-based OS. - -## Installing Dependencies - -When installing Dropbox on the latest modern Distros, the installation of a few dependencies are required before proceeding. Enter the corresponding command for the distro you are using to install these dependencies: - -**Debian and Ubuntu** - - sudo apt install libc6 libglapi-mesa libxdamage1 libxfixes3 libxcb-glx0 libxcb-dri2-0 libxcb-dri3-0 libxcb-present0 libxcb-sync1 libxshmfence1 libxxf86vm1 - -**CentOS** - - yum install tar wget libglapi libXext libXdamage libxshmfence libXxf86vm - - - -## Installing and Configuring Dropbox - -1. Download and install the Dropbox package: - - cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf - - -2. Start the Dropbox daemon: - - ~/.dropbox-dist/dropboxd & - -3. You will receive a message stating that the computer is not linked to your Dropbox account: - - This computer isn't linked to any Dropbox account... - Please visit https://www.dropbox.com/cli_xxxxxxx to link this device. - - Copy the unique URL. Do not copy the one above. - -4. Paste the address above into a web browser and log in to your Dropbox account. You should see the following message in your browser: - - Your computer was successfully linked to your account - - The terminal window on your Linode will show the following message: - - This computer is now linked to Dropbox. Welcome User - - -## Testing the Link - -Any files made within your `Dropbox` directory on your Linode will also be added to Dropbox. - -1. Navigate to your Dropbox folder: - - cd ~/Dropbox - -2. Echo text into a new file: - - echo "testing...." > dropbox-test.txt - -3. Open your Dropbox account in your web browser. `dropbox-test.txt` can now be found in your files! - -Congratulations! Your Linode is now configured to run Dropbox. - - diff --git a/docs/guides/applications/cloud-storage/freenas-blockstorage/index.md b/docs/guides/applications/cloud-storage/freenas-blockstorage/index.md deleted file mode 100644 index 1d8caad044c..00000000000 --- a/docs/guides/applications/cloud-storage/freenas-blockstorage/index.md +++ /dev/null @@ -1,160 +0,0 @@ ---- -slug: freenas-blockstorage -title: "Install FreeNAS on a Linode with Block Storage" -description: "FreeNAS is network-attached storage software configured through a web interface. Learn how to install FreeNAS and connect it to a Block Storage Volume." -authors: ["Edward Angert"] -contributors: ["Edward Angert"] -published: 2018-05-07 -modified: 2018-05-17 -keywords: ["zfs","freenas","block storage","nas"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[FreeNAS Volume Manager Manual](http://doc.freenas.org/11/storage.html)' - - '[Available FreeNAS Plugins](http://doc.freenas.org/11/plugins.html#available-plugins)' -aliases: [] ---- - -Network-attached storage (NAS) allows multiple client devices to access the connected storage media as though it's stored locally to the device. FreeNAS is FreeBSD-based NAS software, configurable via a browser interface. - -This guides shows how to install FreeNAS on a Linode and attach a [Block Storage Volume](https://techdocs.akamai.com/cloud-computing/docs/block-storage) so that you can access both FreeNAS and the Storage Volume from your computer, phone, or tablet almost anywhere in the world. -{{< note type="alert" >}} -FreeNAS is not officially supported by Linode at this time. This means that features like the [Linode Backup Service](https://techdocs.akamai.com/cloud-computing/docs/backup-service) and Lish will be unavailable to you. - -Any issues you may encounter with FreeNAS on your Linode are outside the scope of Linode Support. For further help with this guide's subject, you can ask questions on the [Linode Community Site](https://www.linode.com/community/questions/). -{{< /note >}} - -## Prepare Your Linode - -1. Create a Linode in your preferred data center. Ensure that your Linode has at least 8GB RAM and at least 11GB of available disk space. FreeNAS recommends 16GB of RAM for media servers. Visit the [official requirements](http://www.freenas.org/hardware-requirements/) for more information. - -2. Disable the [Lassie Shutdown Watchdog](https://techdocs.akamai.com/cloud-computing/docs/recover-from-unexpected-shutdowns-with-lassie) to prevent it from attempting to restart your Linode without your input. You can disable Lassie in the **Settings** tab of the Linode Manager under **Shutdown Watchdog**. - -3. [Create two disks](https://techdocs.akamai.com/cloud-computing/docs/manage-disks-on-a-compute-instance#create-a-disk): - - 1. **Label:** Installer - * **Type:** unformatted / raw - * **Size:** 1024 - - 2. **Label:** FreeNAS - * **Type:** unformatted / raw - * **Size:** Can be set to use remaining disk. At least 10240MB - -4. [Create two configuration profiles](https://techdocs.akamai.com/cloud-computing/docs/manage-configuration-profiles-on-a-compute-instance#create-a-configuration-profile) with the following settings. In each profile, disable all of the options under **Filesystem/Boot Helpers**. - - 1. **Label:** Installer - * **Kernel:** Direct Disk - * **/dev/sda:** FreeNAS - * **/dev/sdb:** Installer - * **root / boot device:** Standard /dev/sdb - * **Filesystem/Boot Helpers:** Select No for each Helper - - 2. **Label:** FreeNAS - * **Kernel:** Direct Disk - * **/dev/sda:** FreeNAS - * **root / boot device:** Standard /dev/sda - * **Filesystem/Boot Helpers:** Select No for each Helper - -## Create an Installer Disk - -1. Boot into **Rescue Mode** with the installer disk mounted to `/dev/sda` and access your Linode using [Lish](https://techdocs.akamai.com/cloud-computing/docs/access-your-system-console-using-lish) from the dashboard of your Linode from the Linode Cloud Manager. - -2. Once in Rescue Mode, run the following command to set the [latest FreeNAS release](http://www.freenas.org/download-freenas-release/) (11.1 at the time of this writing) as a variable: - - iso=https://download.freenas.org/11/latest/x64/FreeNAS-11.1-U4.iso - -3. Run the `update-ca-certificates` program to allow the secure download: - - update-ca-certificates - -4. Download the FreeNAS ISO image and expand it to the Installer disk: - - curl $iso | dd of=/dev/sda - -5. When the command finishes, reboot into the **Installer profile**: - - ![Screenshot of the Dashboard, ready to reboot into the Installer profile.](reboot-installer-profile.png "Screenshot of the Dashboard, ready to reboot into the Installer profile.") - -6. Go to the Linode Cloud Manager and access the dashboard for your Linode. - -7. Click the **Launch Console** link to access the [Glish](https://techdocs.akamai.com/cloud-computing/docs/access-your-desktop-environment-using-glish) console and start the installation. - -## Install FreeNAS - -1. Press **Enter** to Install/Upgrade. -2. Press the **Spacebar** to select the FreeNAS disk (`da0` in this screenshot) and press **Enter**: - - ![Select the FreeNAS disk](freenas-installation-select-disk.png "Select the FreeNAS disk.") - -3. The installation shows a warning that all data on the disk will be deleted. Press **Enter** to continue. -4. Enter a root password, tab to the next field to re-enter it, and press **Enter** to continue. -5. Press **Enter** to Boot via BIOS. -6. There's no media to remove. Press **Enter** to acknowledge the successful installation message. -7. Press **4** to select Shutdown System, and press **Enter** to end the session. Close the Glish window. -8. In the Manager, shut the Linode down. - -## Boot and Configure FreeNAS - -1. Select the FreeNAS configuration profile, and click **Boot**: - - ![Screenshot of the Dashboard, ready to reboot into the FreeNAS profile.](reboot-freenas-profile.png "Screenshot of the Dashboard, ready to reboot into the FreeNAS profile.") - - SSH and Lish are disabled in FreeNAS. Use Glish to monitor the first boot which takes several minutes. Once booted, close Glish and proceed to the next step to use the web interface for configuration. - -2. Use a web browser to navigate to the Linode's IP address. Log in with the user `root` and the password set in Step 4 of the previous section. Close any popup menu that appears when you first log in. - -3. Click the **Network** icon and complete the network information using the Default Gateways and DNS Resolvers found in the Networking tab of the Linode Cloud Manager. Use the DNS Resolvers information to fill in the Nameserver fields. Click **Save** before continuing. - -4. Select the **Interfaces** section of the Network tab, and click **Add Interface**. Name the interface and enable DHCP, then click **OK**. - -## Add a Block Storage Volume to FreeNAS - -1. [Add or attach a Block Storage Volume](https://techdocs.akamai.com/cloud-computing/docs/manage-block-storage-volumes) to the Linode. After you attach your Block Storage Volume, the Linode Manager will present command-line instructions for mounting it from your Linode, but you can disregard these. - -2. Reboot the Linode from the Linode Manager. After a few minutes, launch Glish from the dashboard again. You can monitor the reboot progress in Glish. - -### Format Block Storage Volume as ZFS -{{< note type="alert" >}} -Formatting will erase all data on the volume. -{{< /note >}} - -1. Log back into the web interface for FreeNAS. - -2. Click the **Storage** icon at the top, then **View Disks** to confirm that FreeNAS recognized the Block Storage Volume. - -3. Return to the **Storage** tab and click **Volume Manager**. Enter a Volume Name and under Available Disks click the **+** next to the Block Storage Volume. Below Volume layout, select **Stripe**. Click **Add Volume** to format and attach the Volume: - - ![Volume Manager - Add a Volume](freenas-storage-add-volume.png "Volume Manager - Add a Volume") - -### Set Permissions, Share the Volume, and Complete Configuration - -1. Click the Wizard icon: - - ![FreeNAS Wizard Icon](freenas-wizard-icon.png "FreeNAS Wizard Icon") - -2. Select your language, keyboard map, and timezone and click **Next**: - - ![Wizard Keyboard and Timezone Selection](freenas-wizard-lang-keyboard-timezone.png "Wizard Keyboard and Timezone Selection") - -3. When prompted for the Directory Service, press **Next** to skip the step. - -4. Enter a Share name and purpose. Click **Ownership** to configure the Permissions to allow the client to access and manage access. Visit the [official FreeNAS documentation](http://doc.freenas.org/11/storage.html#change-permissions) for more information about the available options. Click **Add**, then **Next** to continue. - -5. Configure e-mail notification settings or press **Next** to skip and continue. - -6. Press **Confirm** and the Wizard will complete the remaining configuration. - -## Enable SSH Root Login (Optional) - -FreeNAS has SSH disabled by default. This is a more secure configuration, but makes it difficult to troubleshoot issues from the command line. - -1. Click the **Services** icon, then the wrench icon on the SSH line: - - ![FreeNAS Services](freenas-services.png "FreeNAS Services") - -2. Check **Login as Root with password** and press **OK**. Check **Start on Boot** if you would like to keep the setting enabled through future reboots. Click **Start Now**. - - Use the same username and password as for the web interface. - -## Next Steps - -Now that FreeNAS is running connected to your Block Storage Volume, you can [connect to it from your local machine](http://doc.freenas.org/11/sharing.html), to a [Plex server](http://www.freenas.org/blog/plex-on-freenas/), or a variety of other platforms using [plugins](http://doc.freenas.org/11/plugins.html#available-plugins). diff --git a/docs/guides/applications/cloud-storage/store-and-share-your-files-with-nextcloud-centos-7/index.md b/docs/guides/applications/cloud-storage/store-and-share-your-files-with-nextcloud-centos-7/index.md deleted file mode 100644 index 0e685aebff9..00000000000 --- a/docs/guides/applications/cloud-storage/store-and-share-your-files-with-nextcloud-centos-7/index.md +++ /dev/null @@ -1,187 +0,0 @@ ---- -slug: store-and-share-your-files-with-nextcloud-centos-7 -title: "Store and Share your Files with Nextcloud on CentOS 7" -description: "Nextcloud is an open content hosting tool built for customization and security. We'll walk you through installing it on CentOS 7." -authors: ["Andrew Lescher"] -contributors: ["Andrew Lescher"] -published: 2017-12-15 -modified: 2018-12-18 -keywords: ["nextcloud", "cloud", "open source hosting"] -tags: ["centos", "lamp"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Using the occ command](https://docs.nextcloud.com/server/12/admin_manual/configuration_server/occ_command.html#http-user-label)' - - '[nginx Configuration](https://docs.nextcloud.com/server/12/admin_manual/installation/nginx.html)' - - '[Enabling SSL](https://docs.nextcloud.com/server/12/admin_manual/installation/source_installation.html#enabling-ssl)' -aliases: [] -relations: - platform: - key: install-nextcloud - keywords: - - distribution: CentOS 7 ---- - -![Store and Share your Files with Nextcloud on CentOS](Store_and_Share_your_Files_with_Nextcloud_on_Centos_smg.png "Store and Share your Files with Nextcloud on CentOS") - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. Install the *EPEL* repository: - - yum install epel-release -y - -## Install MariaDB Database Server - -1. Add the MariaDB 10.2 repository to force yum to install the latest version: - - {{< file "/etc/yum.repos.d/MariaDB.repo" repo >}} -[mariadb] -name = MariaDB-10.2.3 -baseurl = http://yum.mariadb.org/10.2.3/centos7-amd64 -gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB -gpgcheck=1 -{{< /file >}} - -2. Install MariaDB and enable the service on system startup: - - yum install mariadb mariadb-server -y - systemctl start mariadb - systemctl enable mariadb - -3. Set up the MariaDB server with the `mysql_secure_installation` script. Respond to the prompts with the replies shown below: - - mysql_secure_installation - - Enter current password for root (enter for none): ENTER - Set root password? [Y/n] Y - Remove anonymous users? [Y/n] Y - Disallow root login remotely? [Y/n] Y - Remove test database and access to it? [Y/n] Y - Reload privilege tables now? [Y/n] Y - -4. Create a database and user for Nextcloud in MariaDB. Login as `root` and enter the password set earlier. Be sure to create a strong password to replace the `CREATE-PASSWORD-HERE` text: - - mysql -u root -p - - MariaDB [(none)]> CREATE DATABASE nextcloud; - MariaDB [(none)]> GRANT ALL PRIVILEGES ON nextcloud.* TO 'nextclouduser'@'localhost' IDENTIFIED BY 'CREATE-PASSWORD-HERE' WITH GRANT OPTION; - MariaDB [(none)]> FLUSH PRIVILEGES; - MariaDB [(none)]> quit - -## Install Apache Web Server - -1. Install Apache and enable the service on system startup: - - yum install httpd -y - systemctl start httpd - systemctl enable httpd - -2. Disable Apache's WebDAV modules to prevent conflict with Nextcloud's WebDAV modules: - - sudo sed -i 's/^/#&/g' /etc/httpd/conf.modules.d/00-dav.conf - -3. Restart Apache to reflect changes: - - systemctl restart httpd - -## Install PHP 7.1 and Required Modules - -1. Add the Remi repository: - - rpm -Uvh http://rpms.remirepo.net/enterprise/remi-release-7.rpm - -2. Install the *yum-utils* package: - - yum install yum-utils -y - -3. Update the system to populate the Remi repository: - - yum update -y - -4. Direct the system to use PHP 7.1 and issue installation command: - - yum-config-manager --enable remi-php71 - yum install php71-php php-mbstring php-zip php71-php-opcache php71-php-mysql php71-php-pecl-imagick php71-php-intl php71-php-mcrypt php71-php-pdo php-ZendFramework-Db-Adapter-Pdo-Mysql php71-php-pecl-zip php71-php-mbstring php71-php-gd php71-php-xml -y - -5. The default file upload size PHP will allow is 2MB. Increase (or decrease) the allowed file size to your preferred value. The example below will set a 512MB file upload size and no limit for the post size: - - sudo cp /etc/php.ini /etc/php.ini.bak - sudo sed -i "s/post_max_size = 8M/post_max_size = 0/" /etc/php.ini - sudo sed -i "s/upload_max_filesize = 2M/upload_max_filesize = 512M/" /etc/php.ini - -6. Restart Apache: - - systemctl restart httpd - -## Install Nextcloud 12 - -1. Check the [Nextcloud download page](https://nextcloud.com/install/#instructions-server) for the latest version and replace `12.0.4` in the command below with the appropriate version number: - - cd /opt - sudo yum install wget - wget https://download.nextcloud.com/server/releases/nextcloud-12.0.4.zip - -2. Unzip the package: - - sudo yum install unzip - unzip nextcloud-x.y.z.zip - -3. Move the entire unzipped Nextcloud folder to the root web directory and grant permissions to the `apache` user for all contents: - - cp -r nextcloud /var/www/html - -4. Grant permissions to the Nextcloud folder and all its contents to the Apache user. Determine which user Apache is running with the first command below. Replace `apache:apache` in the second command with the output if it differs: - - ps -ef | egrep '(httpd|apache2|apache)' | grep -v `whoami` | grep -v root | head -n1 | groups $(awk '{print $1}') - chown apache:apache -R /var/www/html/nextcloud - -5. Navigate to the `nextcloud` root web directory, and complete the Nextcloud installation: - - cd /var/www/html/nextcloud - sudo -u apache php occ maintenance:install --database "mysql" --database-name "nextcloud" --database-user "nextclouduser" --database-pass "yourpassword" --admin-user "admin" --admin-pass "adminpassword" - -6. If the installation is successful, you will receive the following message: - - Nextcloud was successfully installed - -7. Since these files are now internet-facing, set stronger permissions to improve security: - - find /var/www/html -type f -print0 | sudo xargs -0 chmod 0640 - find /var/www/html -type d -print0 | sudo xargs -0 chmod 0750 - -8. Update the URL in the `config.php` file to accommodate the `nextcloud` subfolder added within the document root. Match the `overwrite.cli.url` and `htaccess.RewriteBase` lines: - - {{< file "/var/www/html/nextcloud/config/config.php" php >}} -. . . - -), - 'datadirectory' => '/var/www/html/nextcloud/data', - 'overwrite.cli.url' => 'http://localhost/nextcloud', - 'htaccess.RewriteBase' => '/nextcloud', - 'dbtype' => 'mysql', - 'version' => '12.0.3.3', - 'dbname' => 'nextcloud', - -. . . -{{< /file >}} - -9. Update the `.htaccess` file with the URL changes: - - sudo -u apache php /var/www/nextcloud/occ maintenance:update:htaccess - -10. Navigate to `your-Linode-IP-address/nextcloud` (replace `your-Linode-IP-address`) and the Nextcloud page should load with a login page for the admin user you created earlier. - - To check the status of your Nextcloud environment, use the following `occ` command: - - sudo -u apache /var/www/html/nextcloud/ php occ status - -## Where to Go from Here - -Once you have successfully installed you Nextcloud environment, you may want to further integrate it into an owned domain name or make adjustments to your web server to serve SSL encrypted pages. See Nextcloud's [Enabling SSL](https://docs.nextcloud.com/server/12/admin_manual/installation/source_installation.html#enabling-ssl) guide to enable SSL. - -Although Apache was used as the web server in this guide, installing Nextcloud with nginx is possible as well. Navigate to the [Nextcloud NGINX Configuration](https://docs.nextcloud.com/server/12/admin_manual/installation/nginx.html) documentation to setup Nextcloud with NGINX. - -[Nextcloud Talk](https://nextcloud.com/talk/), is an addon to Nextcloud that allows for secure text and video conferencing through Nextcloud's platform. Check out our guide on how to [Install Nextcloud Talk](/cloud/guides/install-nextcloud-talk). diff --git a/docs/guides/applications/configuration-management/ansible/front-line-best-practices-ansible/index.md b/docs/guides/applications/configuration-management/ansible/front-line-best-practices-ansible/index.md deleted file mode 100644 index 980003b4cc4..00000000000 --- a/docs/guides/applications/configuration-management/ansible/front-line-best-practices-ansible/index.md +++ /dev/null @@ -1,220 +0,0 @@ ---- -slug: front-line-best-practices-ansible -title: "Best Practices for Ansible" -title_meta: "Top 12 Ansible Best Practices" -description: 'Discover Ansible best practices and proven techniques for project organization, playbook content, documentation, testing, validation, and security.' -authors: ["Cameron Laird"] -contributors: ["Cameron Laird"] -published: 2023-11-28 -keywords: ['ansible best practices','ansible documentation','ansible testing','ansible playbook'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -[Ansible](/cloud/guides/applications/configuration-management/ansible) is an important open source automation tool and platform. It is used for configuration management, application deployment, task automation, and [orchestration](https://www.databricks.com/glossary/orchestration) of complex workflows. - -Ansible figures prominently in DevOps. It allows Information Technology (IT) administrators and developers to automate repetitive tasks and streamline the management and deployment of infrastructure, applications, and services. Ansible’s business and strategic features include: - -- [**Agentless Architecture**](https://www.ansible.com/hubfs/pdfs/Benefits-of-Agentless-WhitePaper.pdf): Does not require installation of agents. -- [**Idempotency**](https://docs.ansible.com/ansible/latest/reference_appendices/glossary.html#term-Idempotency): Gives safe and reliable results from unreliable components. -- [**Portability**](https://www.ansible.com/blog/ansible-and-containers-why-and-how#:~:text=*%20Ansible%20playbooks%20are%20portable.&text=If%20you%20build%20a%20container%20with%20an%20Ansible%20playbook%2C%20you,choice%2C%20or%20on%20bare%20metal.): Operates consistently across different operating systems and into various flavors of cloud environments. - -Data centers effectively require Ansible, or one of its competitors. Businesses operating at the data center scale have requirements for reliability, economy, scalability, and flexibility. These are exactly the advantages of Ansible. It makes data center operations more cost-effective, predictable, resilient, and responsive. - -## Ansible Fundamentals - -The following is a list of key terms that cover the [fundamental components and concepts associated with Ansible](https://docs.ansible.com/ansible/latest/getting_started/basic_concepts.html): - -- **Target State**: Ansible is a [**declarative**](http://www.it-automation.com/2021/06/05/is-ansible-declarative-or-imperative.html) language. It details target states for computing systems and how those states are achieved. It then takes responsibility for achievement of the target states. This creates a kind of [teamwork](https://www.linkedin.com/pulse/delegating-goals-versus-tasks-karl-maier) between users and Ansible, where users take the lead in telling what they want, and Ansible works out the details of how it's done. This is different from older styles of [system administration](/cloud/guides/linux-system-administration-basics) and system administration tools. - - An important aspect of target state is how it applies. Many practitioners have strong experience with Ansible's use in provisioning and deployment, but don't realize it also applies in other automations. While it is good at "spinning up" a new server or updating an existing one, it's also handy for many more uses that aid overall system health. For example, daily checks of certificate expirations, or hourly confirmations that file systems have at least 10% free storage. It only takes a few lines of Ansible to implement these and many other target states and verifications. - -- **Playbooks**: Ansible [playbooks](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_intro.html) are written in [YAML](/cloud/guides/yaml-reference) and define a sequence of steps, or "plays", to execute on a target system or group of systems. Playbooks express desired states for systems and how those states are achieved. Ansible then takes responsibility for achieving those states. That dynamic is Ansible’s fundamental accomplishment. - -- **Modules**: Ansible [modules](https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html) are the building blocks of playbooks. Modules are discrete units of code that enact specific tasks such as package management, file configuration, or launching services. One of Ansible's great assets is its enormous collection of built–in modules and the ability for users to author custom ones. - -- **Tasks**: Ansible [tasks](https://docs.ansible.com/ansible/latest/getting_started/basic_concepts.html#tasks) are individual units within a playbook that call modules to perform specific actions. Tasks execute sequentially on target systems to achieve desired states. - -- **Roles**: Ansible [roles](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_reuse_roles.html) organize and package related playbooks, variables, tasks, and other components into reusable and shareable units. Roles' modularization of configuration promotes reusability across playbooks and projects. - -- **Inventory**: The [inventory](https://docs.ansible.com/ansible/latest/getting_started/basic_concepts.html#inventory) file defines the hosts and systems under consideration. Inventory can be either static or dynamic. It typically includes such information as hostnames, IP addresses, groups, and variables. - -- **Variables**: Ansible allows use of [variables](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_variables.html) that make playbooks more dynamic and flexible. They also feature different scopes, including global, playbook, role, and task. - -- **Facts**: Ansible gathers information about target systems using modules called [facts](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_vars_facts.html). Examples of gathered information include hardware, operating systems, and internet addresses. Playbooks inform the decisions they make with such facts. - -- **Templates**: Ansible [templates](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_module.html) are files structured in [Jinja2 syntax](/cloud/guides/introduction-to-jinja-templates-for-salt) with placeholders. Playbook execution dynamically populates the placeholders with variables. Templates can generate configuration files, scripts, and other Ansible artifacts. - -- **Handlers**: Various specific Ansible events trigger [handlers](https://docs.ansible.com/ansible/latest/getting_started/basic_concepts.html#handlers), typically at the conclusion of a playbook run. A common handler responsibility is to restart services after a configuration change. - -- **Ad-hoc Commands**: Being declarative, Ansible is flexible enough to embed several imperative mechanisms which streamline and simplify particular operations. Its [ad–hoc commands](https://docs.ansible.com/ansible/latest/command_guide/intro_adhoc.html) are indispensable for quick system health checks, troubleshooting, and other isolated remedies. - -## Ansible Best Practices - -While best practices certainly improve run-time efficiency, they also improve organizational efficiency. They can promote teamwork, output reliable results with minimum effort, help onboarding, ease maintenance burdens, and even protect from legal liability. - -As Abelson and Sussman wrote: "[Programs must be written for people to read, and only incidentally for machines to execute.](https://medium.com/javarevisited/epic-programmers-quotes-explained-aed933257b93#:~:text=The%20quote%20implies%20that%20writing,involves%20continuous%20updates%20and%20maintenance.)" In much the same way, the best Ansible playbooks are an ongoing asset for their *human* readers. - -Recognize that Ansible playbooks and related specifications are source, or "[code](https://www.cloudbees.com/blog/configuration-as-code-everything-need-know#)". Like all other sources, they deserve a [version-controlled source code control system](/cloud/guides/introduction-to-version-control) to call home. Think of this as "best practice zero", which precedes the following top 12 best practices for using Ansible. - -### File System Layout - -Organize projects with a consistent file system layout. Separate playbooks from roles, in directories respectively named `playbooks` and `roles`. The result should look similar to the following example project directory structure: - -``` -project/ -├── playbooks/ -│ └── example_playbook.yml -├── roles/ -│ └── example_role/ -│ ├── tasks/ -│ ├── handlers/ -│ ├── templates/ -│ └── … -│── ... -├── group_vars/ -│ ├── all.yml -│ ├── production.yml -│ └── development.yml -│── ... -├── inventory/ -│ ├── production_hosts -│ ├── staging_hosts -│ └── development_hosts -│── ... -├── vault/ -│ ├── secret_file.yml -│ └── … -└── ... -``` - -Spelling of file names and other formal aspects of an Ansible project are purely cosmetic and aren't considered actual Ansible programming. All the more reason to standardize on common practices others have identified and save your team's attention for deeper matters. After all, IT is a collaborative undertaking. - -The point of this best practice is less about the virtue or aesthetics of a directory spelled `playbooks` rather than `playbook`, and more about the benefits of a [common language for the whole team](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3575067/#:~:text=A%20shared%2C%20common%20language%20provides%20a%20focus%20for%20all%20stakeholders.&text=It%20is%20most%20effective%20when,help%20to%20decrease%20project%20costs.). Using these best practices, teams can shift their attention from [worrying about particular details](https://americanexpress.io/yak-shaving/) to thinking more about how to work together toward larger business goals. - -### Ansible Configuration - -Use `ansible.cfg` for global configuration. Define sensible defaults for the inventory and roles paths. Use syntactic comments to document the reasons behind the choices made. - -Explicitly set `forks` to control parallelism. Configure `pipelining` to limit `ssh` operations and increase performance. Configure `ControlPath` to share `ssh` connections. Adjust `timeout` and `poll_interval` to manage timeout of long-running tasks. - -Control verbosity of logging based on actual experience and measurements of the specific playbooks in use. Periodically review the configuration to ensure it is consistent with established policies and goals. - -### Playbook Design and Structure - -Use Roles to modularize playbooks. Refer to [Ansible Galaxy](https://galaxy.ansible.com/ui/) for inspiration regarding useful definitions of Roles. Define your own choices in `requirements.yml`. - -Consider segmenting large and complex playbooks into multiple smaller ones, each with a focus on a specific component or functionality. The resulting bundle of playbooks is likely easier to manage than the original complete one. An alternative way to structure complex playbooks is with [tags](https://galaxy.ansible.com/ui/). Tags effectively disable or enable pieces of a playbook. For instance, it's sometimes beneficial to keep a playbook whole, while controlling distinct pieces within it. - -Separate inventory, configuration, and variable information into environment-specific files. - -Maintain a Vault for *all* sensitive or private information. This includes passwords, certificates, tokens, keys, or any other customer details Ansible needs to know. Consider the alternative of storing sensitive data in a file, and referring to it, rather than coding it into Ansible. - -Periodically review and refactor the playbook structure to keep it fresh and well-aligned with project requirements. - -### Variable Names - -Choose descriptive variable names. For instance, `gateway` rather than `gw`. However, also choose brief names over complicated ones. - -Use [snake case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/#:~:text=Snake%20case%20separates%20each%20word,letters%20need%20to%20be%20lowercase.&text=Snake%20case%20is%20used%20for%20creating%20variable%20and%20method%20names.), for example, `database_account` rather than `DatabaseAccount` or other variations. - -Document the purpose, usage, and range of variables with comments. Examples of particularly useful comments focused on specific variables include: -- `# hostname is case-insensitive, so that 'server1' and 'SERVER1' behave identically` -- `# corpus_account must be qualified: 'name@domain.com' is OK, but 'name' is not` - -Group variables hierarchically. This is likely to result in such names as `database_account`, `database_host`, `database_password`, and `database_priority`. Environment-specific variables deserve meaningful prefixes such as `prod_database_account` or `env_database_account`. - -Avoid shadowing [reserved keywords](https://docs.ansible.com/ansible/latest/reference_appendices/playbooks_keywords.html). Rather than `item` or `serial`, choose `server_item` or `hardware_serial_number`. - -Don't be afraid to make exceptions to these rules when appropriate. For example, abbreviate `gateway` down to `gw` if a particular team has a well-understood, longstanding practice of doing so in other systems and languages beyond Ansible. - -Last but not least, always be consistent across playbooks and projects. - -### Error Handling in Ansible - -Error handling is supremely important. Most of the work accomplished by computing systems is done when things work as intended. However, a good playbook has more lines devoted to responding to failures than for what happens when everything goes right. - -"Handling" encompasses everything from ignoring the error, to logging it, notifying a monitoring system, or launching a diagnostic process. The most important best practice in error-handling is explicit use of `ignore_errors` and `register`. When a particular condition is judged to be non-fatal, mark it with `ignore_errors`, allowing the playbook to continue. Also mark it with an appropriate comment such as `# Error here is non-fatal because ...`. Also handle error conditions conditionally with `when: task_result.failed`. A different error-handling mechanism, `block-rescue-always`, is applicable for resource management and cleaning up problematic states. Take advantage of Ansible's `assert` module. Don’t assume that a particular service is available before starting to use it. Instead, `assert` its availability beforehand. - -Learn Ansible's built-in logging, auditing, debugging, and exit code functionality to handle errors most effectively. - -### Ansible Logging - -Ansible logging has several roles. Learn the essentials by practicing with the log and debug modules. In its simplest form, log can deliver a message during playbook execution through a specification such as: - -``` -- name: Log a single diagnostic - log: - msg: "This is the diagnostic logged at this point" -``` - -Next, learn how to configure `callback_whitelist`, `log_level`, and `log_path` in `ansible.cfg`. Experiment with `log_level` to learn how the categories `CRITICAL`, `DEBUG`, `ERROR`, `INFO`, and `WARNING` apply in your projects. Set them to meet the needs of specific applications and your own preferences. Some administrators like to log everything that might be useful, but only enable `CRITICAL` for daily operations. Others only log diagnostics that are guaranteed to demand response. Either approach, or various alternatives between them, work. It's more important is to be consistent about which style you choose. - -[Ansible's callback plugins](https://docs.ansible.com/ansible/latest/plugins/callback.html) naturally apply to many logging situations. For example, when you want to customize output formats, escalate notifications to email, profile performance metrics during an incident, or otherwise meet logging requirements. - -Timestamp your log entries. Rotate logs to ensure efficient use of storage. Archive logs for auditing and compliance. Treat logs as sensitive information that deserve security controls, so configure access only to users with a need to view them. - -With logging basics in place, consider more sophisticated log management through such aggregators as Elasticsearch, Logstash, Kibana, and Splunk. These help scale your ability to analyze logs. - -Decide on a review policy. No well-founded best practice applies universally in regard to how and when to review logs. It's best to be realistic. If decision-makers believe that logs need to be read, allocate time to do so as an explicit policy, and track the results. - -### Inline Comments - -Comments are important and rewarding, although widely under-used in real-world practice. Each time you write a line of Ansible, ask yourself: what would help me understand the intent of this if I return here six months from now? Ideally, your playbooks should be so simple and idiomatic that their source speaks for itself. The next best thing to that ideal situation is source that's so well-commented that it answers any questions that naturally arise. Always write good comments, and insist that your whole team does, too. - -### READMEs - -Write a `README` for each directory and subdirectory in a project. It could be something as brief as: - -```file {title="README.md"} -# Variables for the Staging environment - -This specification details the Ansible variables which -are specific to actions in the staging environment. -``` - -Other `README` files can be several hundred words about the architecture and design decisions that a particular directory represents. Three natural best practices applicable to `README` files are: - -- Create exactly one `README.md` for each directory in a project. -- Format the contents as well-formed [Markdown](https://www.markdownguide.org/getting-started/). -- Provide high-level "philosophy" and motivation in the README. Leave technical details to source comments. Minimize repetition of the source comments, and instead **refer** to them in `README` files. - -### Playbook Documentation - -Prepare a top-level `README.md` which explains the purpose and use of the playbook. Provide the reader with at least one way to test the playbook. In other words, explain how to do something and what the result should be. Include examples, use cases, and references to relevant documents. Provide a link to your policy on the subject. If some aspect of the playbook is hard to explain, it's even more important to explain it. Use dataflow, state, or entity diagrams, as appropriate. - -List Ansible versions that the playbook is compatible with. Include license and copyright notices in the `README.md`. Review the `README.md` periodically to make sure it aligns with the current state of the playbook. The result is a playbook that is easier to use and maintain correctly, particularly for those not involved in its original creation. - -### Use of Vaults for Sensitive Data - -Encrypt sensitive data, including variables, configurations, task contents, and whole files. However, do *not* encrypt information that is not sensitive. Store sensitive variables in `vars/secrets.yml` and encrypt the file. Reference the encrypted variables in the playbooks as necessary. - -Control access to the Vault with such controls as file systems permissions. Only allow authorized users to decrypt sensitive data, and only with a proper decryption key. Choose strong passwords and encryption keys. Write explicit policies for rotation schedules and practice rotation to ensure that it's correctly executed. - -Passwords must *not* appear in playbooks. Use a credential manager, or at least prompt for necessary passwords to be supplied at runtime. - -### Secure Communication - -Ansible projects generally communicate by way of `ssh`, and `ssh` best practices include the following: - -- Choose strong keys -- Choose strong ciphers -- Secure configurations -- Choose key–based authentication rather than password authentication -- Distribute keys securely -- Minimize agent forwarding -- Define appropriate policies, including a limit on login failures and idle timeouts. -- Consider configuring access-control, restrictions on authentication methods, and lists of allowed users through `sshd_config`. - -Configure Ansible to use SSL for communication between control nodes and managed hosts. Control access to the control node with firewalling and network restrictions. Patch Ansible components regularly and enable two-factor authentication (2FA). - -If your project uses [Ansible APIs](https://docs.ansible.com/ansible/latest/api/index.html), configure communication for TLS. If your project uses [Ansible Tower](https://docs.ansible.com/ansible/latest/reference_appendices/tower.html) or [AWX](https://www.ansible.com/products/awx-project/faq), configure HTTPS. - -### Privilege Escalation and Sudo - -Learn Ansible's [`become` feature](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_privilege_escalation.html), designed expressly for secure privilege escalation. Apply `become` precisely, for a single play at a time, rather than for an entire playbook. Use `become_user` as an additional way to increase the precision of an escalation. Study `become_method` to understand the applicability of different escalation methods. For instance, while `sudo` is the default, a PowerBroker-equipped environment needs to favor `pbrun`. Review escalation uses periodically. - -## Conclusion - -The biggest payoffs in regard to Ansible best practices come from routine, non-technical habits. This includes maintaining playbook version control and accurate documentation, separating secrets from public information, roles from actions, and targets from implementations. Update your Ansible instance through a well-defined [software development lifecycle (SDLC)](https://stackify.com/what-is-sdlc/), and use tools such as [Ansible Lint](https://ansible.readthedocs.io/projects/lint/) appropriately. Make sure every line of code exists for a reason. - -While these habits are not technically deep, with them in place across your teams, Ansible's best practices do pay off. \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/basics/using-mktemp-command/index.md b/docs/guides/applications/configuration-management/basics/using-mktemp-command/index.md deleted file mode 100644 index 441b7f8b8fe..00000000000 --- a/docs/guides/applications/configuration-management/basics/using-mktemp-command/index.md +++ /dev/null @@ -1,194 +0,0 @@ ---- -slug: using-mktemp-command -title: "Using mktemp Command to Create Temporary Files and Directories" -title_meta: "How to Use the mktemp Command to Create Temporary Files and Directories" -description: 'Learn how to use the mktemp command on an Ubuntu 20.04 Linode server. Using the mktemp command, you can create temporary files and directories.' -authors: ["Tom Henderson"] -contributors: ["Tom Henderson"] -published: 2023-03-19 -keywords: ['mktemp', 'mktemp bash', 'mktemp directory', 'tmpdir'] -tags: ['linux'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Setting and Using Linux Environment Variables](/cloud/guides/how-to-set-linux-environment-variables)' ---- - -The `mktemp` command is used in Linux and BSD derivative operating systems to create temporary files or directories. The temporary filename and directories can be named using a user-defined "template". This utility is installed by default on major Linux distributions. - -Using `mktemp` varies from a traditional method of naming files using the PID in scripts. The use of `mktemp` command allows a naming convention in a script that can be controlled by the script, and is more unlikely to collide with an existing temporary file, thus avoiding inadvertent temp file overwriting, or other undesirable collisions. - -This guide demonstrates where `mktemp` directives can be useful, and how to use templates to direct the formation of the file names. - -## Linux Temporary File and Directories: Use Cases - -Temporary files created by `mktemp` are owned by the user; commonly called from scripts to store data, in and among apps used within the script. It is a good practice to create temporary files in the `/tmp` directory, which is created by the operating system. - -The `mktemp` command is different from the `touch` command. `mktemp` creates files or directories that are flexible. It has features to ensure the file names are unique, generated randomly, and reduces accidental overwrite, or incorrect manipulation of an existing file, or directory. The `touch` command modifies the timestamp of a file and is traditionally used to create permanent files, while `mktemp` is used to create temporary files. - -A script or independent invocation of `mktemp` can also use a template that forms the names of the temp files and/or directories started by `mktemp`. - -**Use Case #1 - Create A Simple Temporary File** - -Invoking `mktemp` with no arguments creates a random file in the `/tmp` directory. The `/tmp` directory is the default directory for file creations unless specified otherwise. - -```command -mktemp -``` -```output -/tmp/tmp.df8N4EE9Y -``` - -**Use Case #2 - Create A Simple Temporary Directory** - -The `mktemp` command with the `-d` argument creates a temporary directory in the `/tmp` directory. - -```command -mktemp -d -``` -```output -/tmp/tmp.df8N4EE9Y -``` - -By default, `mktemp` command generates a random name that can be customized by passing a template argument. The template requires a minimum of three 'X' characters to be specified which indicates the places to be occupied by random characters. - -```command -tmp_dir=$(mktemp -d -t test-XXXX) -echo $tmp_dir -``` -```output -/tmp/test-Xsd2ewsd -``` - -The `mktemp` command uses the currently set environment variable `$TMPDIR` to place a new temporary directory. The default `$TMPDIR` value found by most shell scripts is the `/tmp` directory, or the other choice is the `/var/tmp` directory. The difference between `/tmp` and `/var/tmp` is that the data stored in `/var/tmp` directory is preserved between reboots and is more persistent than the data in `/tmp`. `/var/tmp` is not usually subjected to `systemctl` extensions that may control temporary files. - -### How mktemp Is Used By System Administrators - -The `mktemp` command declares an explicit file or directory that is meant to be temporary. Although the `/tmp` directory is often used for temporary files, applications used by multiple users and/or similar processes may create files using the same filename. For example, a single process can execute independently to create a file that can read, write, update, or delete the same file. This may cause a filename conflict requiring an administrative effort to rectify the problem. But, when a script calls `mktemp`, a unique filename is created that is unlikely to collide with an existing filename or directory having the same name. - -You can use `mktemp` independently or within scripts to: - -- Create temporary files and/or directories -- Test whether temporary file and/or directory creation is possible -- Create file or directory names that follow a specific identity pattern with added random characters generated from an invocation-assigned pattern -- Change the `TMPDIR` environmental variable to a custom directory other than the default `/tmp`, with an optional pattern generated from an invocation-assigned pattern. -- Keep temporary files and directories protected as to the ownership of the user rights spawning the invocation of the command, so that other uses of the temporary files and directories created are protected from other scripts, processes, and filename collision caused by other users. - -## How to Create a Temporary File - -The exact `mktemp` syntax is described in the [man(mktemp) pages](https://man7.org/linux/man-pages/man3/mktemp.3.html). Most POSIX-compliant systems use the same syntax as shown in the table below: - - -| Command | Description | -| ----------------------------- |-----------------------------------------------------------------------------| -| `mktemp` | Makes a temporary file in the default temporary directory | -| `mktemp -V` | Display mktemp version then exit | -| `mktemp -d` | Make a directory in the default temporary directory | -| `mktemp -p` | (implies `-t`) Uses the as a prefix; the `-t` option generates a path in the default temporary directory (as specified by the environmental variable `$TMPDIR`). | -| `mktemp -q` | Perform execution of the command quietly, meaning without success or failure messages | -| `mktemp -t` | Make a temporary filename, using a template, to be located in `/tmp` directory unless the `$TMPDIR` directory exists, and if it exists, make the file in the directory specified by the `$TMPDIR` value | - -## How to Create a Temporary Directory - -When `mktemp` is invoked with the `-d` argument, it creates a directory in `/tmp`. If there is an environmental variable `$TMPDIR`, then it creates a directory in the `$TMPDIR` directory. - -```command -mktemp -d -``` -```output -/tmp/tmp.sdasdas -``` - - -### How to Create a Temporary File Template - -A temporary file template can be used to differentiate the source or criteria by naming the file with a template. The `mktemp` template uses the letter "X" to be replaced by a random character in place of the “X” when used as an argument. Up to ten “X” characters in a row, for a total of ten places can be randomized in this way. - -The following command is used to create a file template, where "X" is replaced by randomized characters: - -```command -mktemp -t random-XXXXXXXX -``` -```output -/var/folders/gh/dbgxkdts6414dlmnflxbp2h80000gn/T/random-XXXXXXXX.4b5Dfmg4 -``` - -The following command which uses the `-d` argument to `mktemp` produces a directory in the same way. - -```command -mktemp -d random-XXXXXXXX -``` -```output -random-13EL42PM -``` - -You can view the above newly created temporary directory using the `ls -la` command. - -You can also add a suffix to a template while creating the directory as shown in the command below: - -```command -mktemp -d --suffix TODAY -``` -```output -/tmp/tmp.lmnflxbTODAY -``` - -## The TMPDIR Environment Variable - -The `TMPDIR` environmental variable enables you to specify a different path for you to store the temporary files. It is stored in a list that is available to applications and shell scripts. The `TMPDIR` variable permits many applications to know where the administrator has designated the storage of temporary directories, especially if the designation varies from the default use of the `/tmp` directory. The `/tmp` directory in some instances may be placed on special media like an SSD for speed purposes. To understand more on environment variables, see the Linode's guide on [Setting and Using Linux Environment Variables](/cloud/guides/how-to-set-linux-environment-variables). - -On some Linux systems, the `TMPDIR` file is called or declared by `systemd-tempfiles`, a daemon that can be set to periodically clean files by creation date, or other attributes not covered in this guide. - -If the `TMPDIR` variable is changed, its value may only survive for the current session of the user or PID. When you reboot the system, it may default to `/tmp`, or the setting called by the `systemd-tempfiles.conf` file. When you restart a system or a session, or any other event, the `TMPDIR` is restored to its previous value. - -The `mktemp --tmpdir` argument changes the destination relative to the value set by the `$TMPDIR` value set in the environment. - -In the example command below, the `$TMPDIR` value is changed to a subdirectory, and `mktemp` applies its files to the new path: - -```command -$TMPDIR=(mktemp -d) -``` - -Template and other arguments could also be added to change the `$TMPDIR` value. An example of a date-codified directory is shown below. - -```command -root@localhost:/home# tmpdir=$(mktemp -d -t ci-$(date +%Y-m-%d-%H-%M-%S)-XXXXXXXXXX) -root@localhost:/home# echo $tmpdir -``` -```output -/var/folders/gh/dbgxkdts6414dlmnflxbp2h80000gn/T/ci-2022-m-28-22-46-25-XXXXXXXXXX.6lR6R7Az -``` - -## How to Delete Your Linux System’s Temp Files - -Cleaning the temporary files depends on the Linux version you are using. Current Linux systems using *systemd* use a process called *systemd-tempfiles*. Depending on the system version, and its implementation, the files, and process used for cleaning the temporary files, and directories have different configurations. - -It is a good practice to delete the temporary files in the `/tmp` directory of your system frequently. This takes up unnecessary space that could be used for other data or processes. Generally, files in the `/tmp` directory are removed by your system after every reboot. - -{{< note >}} -The temporary files in the `/var/tmp` directory are usually preserved between system reboots and are made available to the programs that require temporary files. The data stored in the `/var/tmp` is more persistent that the data in the `/tmp` directory. -{{< /note >}} - -The following section describes the different ways in which you can delete temporary files from your system. - -For the currently logged-in user, where the user has no other active processes, the `/tmp` directory is deleted by invoking the following command: - -```command -rm -rf /tmp -``` - -This deletes all `/tmp` files recursively through subdirectories and forces the deletion of all files for which the user has privileges. If the user is the root or sudo, then all files may be deleted, and this can be disruptive. Depending on the currently logged-in user rights, the `rm -rf /tmp` command may destroy files and directories that are otherwise in use. - -You can also use the prebuilt `find` command that is available in almost every Linux distribution. The `find` command allows you to find the files and directories that satisfy a specific condition. For example, the following `find` command finds and deletes all the temp files in the `/tmp` directory. - -```command -sudo find /tmp -type f -delete -``` - -The `trap` command can be used to manage the deletion of files made within a script. When the shell has finished its execution, the `trap` command allows the specified temporary files to be deleted. - -For example, if your script creates a temporary file and you want to delete it at each place where you exit your script, you can include a `trap` command at the start of your script that deletes the file on exit: - -```file -tempfile=/tmp/tmpdata -trap "rm -f $tempfile" EXIT -``` diff --git a/docs/guides/applications/configuration-management/chef/how-to-install-chef-on-ubuntu-20-04/index.md b/docs/guides/applications/configuration-management/chef/how-to-install-chef-on-ubuntu-20-04/index.md deleted file mode 100644 index e988c73be67..00000000000 --- a/docs/guides/applications/configuration-management/chef/how-to-install-chef-on-ubuntu-20-04/index.md +++ /dev/null @@ -1,569 +0,0 @@ ---- -slug: how-to-install-chef-on-ubuntu-20-04 -title: "Install Chef on Ubuntu 20.04" -title_meta: "How to Install Chef on Ubuntu 20.04" -description: 'This guide provides a brief introduction to the Chef configuration management software and explains how to install and use it.' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2022-10-17 -modified: 2022-11-28 -keywords: ['Install Chef','Configure Chef','Chef Ubuntu','Chef Server','Chef Workstation'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Chef Website](https://www.chef.io/)' -- '[Chef Overview](https://docs.chef.io/platform_overview/)' -- '[Chef Workstation documentation](https://docs.chef.io/workstation/)' -- '[Learn Chef training courses](https://learn.chef.io/)' -- '[Chef Workstation Downloads for Ubuntu](https://www.chef.io/downloads/tools/workstation?os=ubuntu)' -- '[Chef Workstation installation instructions](https://docs.chef.io/workstation/install_workstation/)' -- '[Chef Server Downloads for Ubuntu](https://www.chef.io/downloads/tools/infra-server?os=ubuntu)' -- '[Chef Server installation instructions](https://docs.chef.io/server/install_server/)' -- '[Chef Supermarket](https://supermarket.chef.io/)' -- '[Chef Supermarket documentation](https://docs.chef.io/workstation/knife_supermarket/)' -- '[Chef bootstrap documentation](https://docs.chef.io/install_bootstrap/)' ---- - -[Chef](https://www.chef.io/) is a free and open source *Infrastructure as Code* (IaC) application. It's a configuration management system that allows administrators to provision and manage infrastructure using automation. A complete Chef workflow includes one or more Chef Workstations, a Chef Server, and a set of nodes. This guide provides some background on how Chef works, and explains how to install and configure Chef on Ubuntu 20.04. - -## What is Chef? - -Chef is a IaC application for automating and streamlining the process of provisioning, configuring, deploying, and managing network nodes. It allows for continuous deployment and an automated environment. Chef can manage many types of components including servers, containers, and networking infrastructure. - -Chef operates using a hub-and-spoke architecture, with the master Chef Server at the center. One or more Chef Workstations interact with the Server, which automates the configuration of one or more Chef nodes. Configuration assets move from the workstation to the server and finally to the nodes. Workstations cannot interact with the nodes directly. The Chef infrastructure consists of the following components. - -- **Chef Workstation**: A workstation is a server for creating and testing configuration code. The code is then pushed to the Chef Server. Several workstations can interact with the same server, but each workstation only links to one server. The [Chef Workstation documentation](https://docs.chef.io/workstation/) contains more information on how to use the workstation. -- **Chef Server**: The Chef Server is the "command center" for the entire system. It stores and maintains all the configuration files, code, and scripts. A Chef Server includes many components, including a web server and PostgreSQL database. It is responsible for pushing the relevant assets to the various nodes and keeping track of the nodes under its management. Each server is highly capable, efficient, and robust, and is capable of managing a large number of nodes. -- **Chef Node**: The Chef Server deploys and manages a node using assets developed on the Chef Workstation. Each node is administered by a single Chef Server. Although it is dependent on the server, a Chef Node contains a Chef client. The client queries the server for updates and keeps the node up to date. - -The following illustration indicates the relationship between the three parts of the Chef system. - -![Chef Components](Chef-Components.png) - -The workstations use Chef commands, such as the `knife` directive, to interact with the server. Chef incorporates extra security and authentication into all of its operations, using public key encryption. However, the Chef system is complex and has a high learning curve. - -Chef uses an idiosyncratic terminology based on cooking vocabulary. Some of the more important terms include the following: - -- **Attribute**: Specifies a value for an item on a node. -- **Bookshelf**: Stores the various cookbooks and assets on a Chef Server using versioning control. -- **Chef-client**: Runs on the node, and is responsible for verifying whether the node is up-to-date with the assets stored on the server. -- **Chef-repo**: A directory on the Chef Workstation that contains the local cookbooks and configuration files. -- **Cookbook**: The primary method of managing nodes. It contains information describing the final state of a node. The Chef server and node use the cookbook to guide configuration. Cookbooks contain *recipes*, along with attributes, libraries, templates, and scripts. These cookbooks can be developed on the workstation or downloaded from the [Chef Supermarket](https://supermarket.chef.io/). -- **Environment**: Collects nodes into groups to better organize them. Similar configurations and scripts can be applied to the entire group. -- **Knife**: A Chef Workstation uses the `knife` tool to correspond with the Chef Server. A `knife` command usually takes the format `knife subcommand [ARGUMENT] (options)`. -- **Recipe**: A recipe is contained within a cookbook. It explains the resources to add, change, or run on the node. Recipes are written in Ruby. -- **Resource**: A resource is part of a recipe. It contains a type, name, and list of key-value pairs for a component. -- **Test Kitchen**: This is a workstation module to help users test recipes before deployment. - -Linode has a helpful [Beginner's Guide to Chef](/cloud/guides/beginners-guide-chef). For detailed information about Chef, see the [Chef documentation](https://docs.chef.io/). Chef also makes the [Learn Chef](https://learn.chef.io/) training resource available. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. At least three Linode systems running Ubuntu 20.04 are required to implement a Chef system. One server is for the Chef Workstation, the second for the Chef Server, while a third represents a node under administration. Due to Chef's memory demands, the Chef Server requires a 8GB Linode. The other two servers can be 2GB Linodes. Both the Chef Server and Chef Workstation should be configured using the previous instructions. Chef is used to set up the target node. - -1. Ensure all Linode servers are updated using the following command. - - ```command - sudo apt update && sudo apt upgrade - ``` - -1. Assign a domain name to the Chef Server. For information on domain names and pointing the domain name to a Linode, see the [Linode DNS Manager guide](https://techdocs.akamai.com/cloud-computing/docs/dns-manager). - -1. Configure the host name of the Chef Server so it matches the domain name. This allows SSL certificate allocation to proceed normally. To set the host name of a Ubuntu server, use the command `sudo hostnamectl set-hostname `, replacing `` with the actual name of your domain. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you are not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## How to Install and Configure the Chef Server - -Because the Chef Server operates as a hub for the entire system, it is best to install and configure it first. The Chef Server uses a high amount of resources, so it is important to use a dedicated Linode with at least 8GB of memory. - -### How to Install the Chef Server - -The Chef Server Core can be downloaded using `wget`. The following steps demonstrate how to download the latest release of Chef for the Ubuntu 20.04 release. For other releases of Ubuntu, see the [Chef download page](https://www.chef.io/downloads/tools/infra-server?os=ubuntu). For more detailed instructions, see the [Chef Server installation page](https://docs.chef.io/server/install_server/). To install the Chef Server, follow these steps. - -1. Download the Chef Server core using `wget`. - - ```command - wget https://packages.chef.io/files/stable/chef-server/15.1.7/ubuntu/20.04/chef-server-core_15.1.7-1_amd64.deb - ``` - -1. Install the server core. - - ```command - sudo dpkg -i chef-server-core_*.deb - ``` - - ```output - Selecting previously unselected package chef-server-core. - (Reading database ... 108635 files and directories currently installed.) - Preparing to unpack chef-server-core_15.1.7-1_amd64.deb ... - Unpacking chef-server-core (15.1.7-1) ... - Setting up chef-server-core (15.1.7-1) ... - Thank you for installing Chef Infra Server! - ``` -1. For better security and to preserve server space, remove the downloaded `.deb` file. - - ```command - rm chef-server-core_*.deb - ``` - -1. Start the Chef server. Answer `yes` when prompted to accept the product licenses. - - {{< note respectIndent=false >}} -The installation process takes several minutes to complete. Upon a successful installation, the message `Chef Infra Server Reconfigured!` is displayed. - {{< /note >}} - - ```command - sudo chef-server-ctl reconfigure - ``` - -### How to Configure a Chef User and Organization - -To use Chef, configure an organization and at least one user on the Chef Server. This enables server access for workstations and nodes. To create these accounts, follow these steps. - -1. Create a `.chef` directory to store the keys. This should be a subdirectory located inside the home directory. - - ```command - mkdir .chef - ``` - -1. Use the `chef-server-ctl` command to create a user account for the Chef administrator. Additional user accounts can be created later. Replace the `USER_NAME`, `FIRST_NAME`, `LAST_NAME`, `EMAIL`, and `PASSWORD` fields with the relevant information. For the `--filename` argument, replace `USER_NAME.pem` with the user name used earlier in the command. - - ```command - sudo chef-server-ctl user-create USER_NAME FIRST_NAME LAST_NAME EMAIL 'PASSWORD' --filename ~/.chef/USER_NAME.pem - ``` - -1. Review the user list and confirm the account now exists. - - ```command - sudo chef-server-ctl user-list - ``` - - ```output - USER_NAME - ``` - -1. Create a new organization, also using the `chef-server-ctl` command. Replace `ORG_NAME` and `ORG_FULL_NAME` with the actual name of the organization. The `ORG_NAME` field must be all lower case. The value for `USER_NAME` must be the same name used in the `user-create` command. For the `--filename` argument, in `ORG_NAME.pem`, replace `ORG_NAME` with the organization name used elsewhere in the command. - - ```command - sudo chef-server-ctl org-create ORG_NAME "ORG_FULL_NAME" --association_user USER_NAME --filename ~/.chef/ORG_NAME.pem - ``` - -1. List the organizations to confirm the new organization is successfully created. - - ```command - sudo chef-server-ctl org-list - ``` - - ```output - ORG_NAME - ``` - -## How to Install and Configure a Chef Workstation - -A Chef Workstation is for users to create and test recipes. Any Linode with at least 2GB of memory can be used for this task. Unlike the Chef Server, a workstation can also be used for other tasks. However, in a larger organization hosting many users, it is often efficient to centralize workstation activities on one server hosting multiple accounts. - -### How to Install a Chef Workstation - -The steps for installing a Chef Workstation are similar to those for installing the Server. Download the correct file using `wget`, then install it. To install a Chef Workstation, follow these steps. - -1. Download the source files for the Chef Workstation. For different releases of the Workstation, or downloads for earlier releases, see the [Chef Workstation Downloads page](https://www.chef.io/downloads/tools/workstation?os=ubuntu). For more information on the installation process, see the [Chef Workstation Installation Documentation](https://docs.chef.io/workstation/install_workstation/). - - ```command - wget https://packages.chef.io/files/stable/chef-workstation/22.10.1013/ubuntu/20.04/chef-workstation_22.10.1013-1_amd64.deb - ``` - -1. Install the Chef Workstation. - - ```command - sudo dpkg -i chef-workstation_*.deb - ``` - - ```output - Thank you for installing Chef Workstation! - ``` - -1. Remove the source file. - - ```command - rm chef-workstation_*.deb - ``` - -1. Confirm the correct release of the Chef Workstation is installed. - - ```command - chef -v - ``` - - ```output - Chef Workstation version: 22.10.1013 - Chef Infra Client version: 17.10.0 - Chef InSpec version: 4.56.20 - Chef CLI version: 5.6.1 - Chef Habitat version: 1.6.521 - Test Kitchen version: 3.3.2 - Cookstyle version: 7.32.1 - ``` - -### How to Configure a Chef Workstation - -A few more items must be configured before the Workstation is operational. Tasks include generating a repository, editing the `hosts` file, and creating a subdirectory. To fully configure the workstation, follow these steps. - -1. Generate the `chef-repo` repository. This directory stores the Chef cookbooks and recipes. Enter `yes` when asked whether to accept the product licenses. - - ```command - chef generate repo chef-repo - ``` - - ```output - Your new Chef Infra repo is ready! Type `cd chef-repo` to enter it. - ``` - -1. Edit the `/etc/hosts` file. This file contains mappings between host names and their IP addresses. Add an entry for the Chef Server, containing the name of the server, which is also the domain name, and its IP address. In this example, this is indicated in the line `192.0.1.0 example.com`. There must also be an entry for the local server. This is the `192.0.2.0 chefworkstation` line in the example. This entry must contain the local IP address and the hostname of the server hosting the Chef Workstation. The file should resemble the following example. - - ```file {title="/etc/hosts" lang="conf"} - 127.0.0.1 localhost - 192.0.1.0 example.com - 192.0.2.0 chefworkstation - ``` - -1. Create a `.chef` subdirectory. This is where the `knife` file is stored, along with files for encryption and security. - - ```command - mkdir ~/chef-repo/.chef - cd chef-repo - ``` - -### How to Add RSA Private Keys - -RSA private keys enable better security between the Chef Server and associated workstations through the use of encryption. Earlier, RSA private keys were created on the Chef Server. Copying these keys to a workstation allows it to communicate with the server. To enable encryption using RSA private keys, follow these steps. - -{{< note >}} -SSH password authentication must be enabled on the Chef Server to complete the key exchange. If SSH password authentication has been disabled for better security, enable it again before proceeding. After the keys have been retrieved and added to the workstation, SSH password authentication can be disabled again. See the Linode guide to [How to Secure Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#ssh-daemon-options) for more information. -{{< /note >}} - -1. On the workstation, generate an RSA key pair. This key can be used to initially access the Chef server to copy over the private encryption files. - - ```command - ssh-keygen -b 4096 - ``` - - ```output - Generating public/private rsa key pair. - Enter file in which to save the key (/home/username/.ssh/id_rsa): - ``` - -1. Hit the **Enter** key to accept the default file names `id_rsa` and `id_rsa.pub`. Ubuntu stores these files in the `/home/username/.ssh` directory. - - ```output - Created directory '/home/username/.ssh'. - Enter passphrase (empty for no passphrase): - ``` - -1. Enter a password when prompted, then enter it again. An identifier and public key are saved to the directory. - - ```output - Your identification has been saved in /home/username/.ssh/id_rsa - Your public key has been saved in /home/username/.ssh/id_rsa.pub - ``` - -1. Copy the new public key from the workstation to the Chef Server. In the following command, use the account name for the Chef Server along with its IP address. - - ```command - ssh-copy-id username@192.0.1.0 - ``` - -1. Use the `scp` command to copy the `.pem` files from the Chef Server to the workstation. In the following example, replace `username` with the user account for the Chef Server and `192.0.1.0` with the actual Chef Server IP address. - - ```command - scp username@192.0.1.0:~/.chef/*.pem ~/chef-repo/.chef/ - ``` - - ```output - Enter passphrase for key '/home/username/.ssh/id_rsa': - username.pem 100% 1674 1.7MB/s 00:00 - testcompany.pem 100% 1678 4.7MB/s 00:00 - ``` - -1. List the contents of the `.chef` subdirectory to ensure the `.pem` files were successfully copied. - - ```command - ls ~/chef-repo/.chef - ``` - - ```output - username.pem testcompany.pem - ``` - -### How to Configure Git on a Chef Workstation - -A *version control system* helps the Chef Workstation track any changes to the cookbooks and restore earlier versions if necessary. This example uses Git, which is compatible with the Chef system. The following steps explain how to configure Git, initialize a Git repository, add new files, and commit them. - -1. Configure Git using the `git config` command. Replace `username` and `user@email.com` with your own values. - - ```command - git config --global user.name username - git config --global user.email user@email.com - ``` - -1. Add the `.chef` directory to the `.gitignore` file. This ensures system and auto-generated files are not shown in the output of `git status` and other Git commands. - - ```command - echo ".chef" > ~/chef-repo/.gitignore - ``` - -1. Ensure the `chef-repo` directory is the current working directory. Add and commit the existing files using `git add` and `git commit`. - - ```command - cd ~/chef-repo - git add . - git commit -m "initial commit" - ``` - - ```output - [master (root-commit) a3208a3] initial commit - 13 files changed, 343 insertions(+) - create mode 100644 .chef-repo.txt - ... - create mode 100644 policyfiles/README.md - ``` - -1. Run the `git status` command to ensure all files have been committed. - - ```command - git status - ``` - - ```output - On branch master - nothing to commit, working tree clean - ``` - -## How to Generate a Chef Cookbook - -To generate a new Chef cookbook, use the `chef generate` command. - -```command -chef generate cookbook my_cookbook -``` - -## How to Configure the Knife Utility - -The Chef Knife utility helps a Chef workstation communicate with the server. It provides a method of managing cookbooks, nodes, and the Chef environment. Chef uses the `config.rb` file in the `.chef` subdirectory to store the Knife configuration. To configure Knife, follow these steps. - -1. Create a `config.rb` file in the `~/chef-repo/.chef` directory. This example uses `vi`, but any text editor can be used. - - ```command - cd ~/chef-repo/.chef - vi config.rb - ``` - -1. Use the following `config.rb` file as an example of how to configure Knife. Copy this sample configuration to the file. - - ```file {title="~/chef-repo/.chef/config.rb" lang="ruby"} - current_dir = File.dirname(__FILE__) - log_level :info - log_location STDOUT - node_name 'node_name' - client_key "USER.pem" - validation_client_name 'ORG_NAME-validator' - validation_key "ORG_NAME-validator.pem" - chef_server_url 'https://example.com/organizations/ORG_NAME' - cache_type 'BasicFile' - cache_options( :path => "#{ENV['HOME']}/.chef/checksums" ) - cookbook_path ["#{current_dir}/../cookbooks"] - ``` - -1. Make the following changes: - - - `node_name` must be the name of the user account created when configuring the Chef Server. - - For `client_key`, replace `USER` with the user name associated with the `.pem` file, followed by `.pem`. - - `validation_client_name` requires the same `ORG_NAME` used when creating the organization followed by `-validator`. - - the `validation_key` field must contain the name used for `ORG_NAME` when the organization was created, followed by `-validator.pem`. - - For `chef_server_url`, change `example.com` to the name of the domain. Follow the domain name with `/organizations/` and the `ORG_NAME` used when creating the organization. - - Leave the remaining fields unchanged. - -1. Move back to the `chef-repo` directory and fetch the necessary SSL certificates from the server using the `knife fetch` command. - - {{< note respectIndent=false >}} -The SSL certificates were generated when the Chef server was installed. The certificates are self-signed. This means a certificate authority has not verified them. Before fetching the certificates, log in to the Chef server and ensure the hostname and fully qualified domain name (FQDN) are the same. These values can be confirmed using the commands `hostname` and `hostname -f`. - {{< /note >}} - - ```command - cd .. - knife ssl fetch - ``` - - ```output - Knife has no means to verify these are the correct certificates. You should verify the authenticity of these certificates after downloading. - Adding certificate for example.com in /home/username/chef-repo/.chef/trusted_certs/example.com.crt - ``` - -1. To confirm the `config.rb` file is correct, run the `knife client list` command. The relevant validator name should be displayed. - - ```command - knife client list - ``` - - ```output - testcompany-validator - ``` - -## How to Bootstrap a Node - -At this point, both the Chef Server and Chef Workstation are configured. They can now be used to bootstrap the node. The bootstrap process installs the chef client on the node and performs validation. The node can then retrieve any necessary updates from the Chef Server. To bootstrap the node, follow these steps. - -1. Log in to the target node, which is the node to be bootstrapped, and edit the `/etc/hosts` file. Add entries for the node, the Chef server domain name, and the workstation. The file should resemble the following example, using the actual names of the Chef Server, workstation, and the target node, along with their IP addresses. - - ```file {title="/etc/hosts" lang="conf"} - 127.0.0.1 localhost - 192.0.100.0 targetnode - 192.0.2.0 chefworkstation - 192.0.1.0 example.com - ``` - -1. Return to the Linode hosting the Chef Workstation and change the working directory to `~/chef-repo/.chef`. - - ```command - cd ~/chef-repo/.chef - ``` - -1. Bootstrap the node using the `knife bootstrap` command. Specify the IP address of the target node for `node_ip_address`. This is the address of the node to bootstrap. In the following example, use the actual user name and password for the account in place of `username` and `password`. Enter the name of the node in place of `nodename`. Answer `Y` when asked "Are you sure you want to continue connecting". - - {{< note respectIndent=false >}} -The option to bootstrap using key-pair authentication no longer appears to be supported. - {{< /note >}} - - ```command - knife bootstrap node_ip_address -U username -P password --sudo --use-sudo-password --node-name nodename - ``` - -1. Confirm the node has been successfully bootstrapped. List the client nodes using the `knife client list` command. All bootstrapped nodes should be listed. - - ```command - knife client list - ``` - - ```output - target-node - testcompany-validator - ``` - -1. Add the bootstrapped node to the workstation `/etc/hosts` file as follows. Replace `192.0.100.0 targetnode` with the IP address and name of the bootstrapped node. - - ```file {title="/etc/hosts" lang="conf"} - 127.0.0.1 localhost - 192.0.1.0 example.com - 192.0.2.0 chefworkstation - 192.0.100.0 targetnode - ``` - -## How to Download and Apply a Cookbook (Optional) - -A cookbook is the most efficient way of keeping target nodes up to date. In addition, a cookbook can delete the `validation.pem` file that was created on the node when it was bootstrapped. It is important to delete this file for security reasons. - -It is not mandatory to download or create cookbooks to use Chef. But this section provides a brief example of how to download a cookbook and apply it to a node. - -1. On the Chef workstation, change to the `~/chef-repo/.chef` directory. - - ```command - cd ~/chef-repo/.chef - ``` - -1. Download the `cron-delvalidate` cookbook from the Chef Supermarket. For more information on the `supermarket` command see the [Chef supermarket documentation](https://docs.chef.io/workstation/knife_supermarket/). - - ```command - knife supermarket download cron-delvalidate - ``` - - ```output - Downloading cron-delvalidate from Supermarket at version 0.1.3 to /home/username/chef-repo/.chef/cron-delvalidate-0.1.3.tar.gz - Cookbook saved: /home/username/chef-repo/.chef/cron-delvalidate-0.1.3.tar.gz - ``` - -1. If the cookbook is downloaded as a `.tar.gz` file, use the `tar` command to extract it. Move the extracted directory to the `cookbooks` directory. - - ```command - tar -xf cron-delvalidate-0.1.3.tar.gz - cp -r cron-delvalidate ~/chef-repo/cookbooks/ - ``` - -1. Review the cookbook's `default.rb` file to see the recipe. This recipe is written in Ruby and demonstrates how a typical recipe is structured. It contains a cron job named `clientrun`. This job instantiates a new cron job to run the `chef-client` command on an hourly basis. It also removes the extraneous `validation.pem` file. - - ```file {title="~/chef-repo/cookbooks/cron-delvalidate/recipes/default.rb" lang="ruby"} - # - # Cookbook Name:: cron-delvalidate - # Recipe:: Chef-Client Cron & Delete Validation.pem - # - # - cron "clientrun" do - minute '0' - hour '*/1' - command "/usr/bin/chef-client" - action :create - end - - file "/etc/chef/validation.pem" do - action :delete - end - ``` - -1. Add the recipe to the run list for the node. In the following command, replace `nodename` with the name of the node. - - ```command - knife node run_list add nodename 'recipe[cron-delvalidate::default]' - ``` - - ```output - nodename: - run_list: recipe[cron-delvalidate::default] - ``` - -1. Upload the cookbook and its recipes to the Chef Server. - - ```command - knife cookbook upload cron-delvalidate - ``` - - ```output - Uploading cron-delvalidate [0.1.3] - Uploaded 1 cookbook. - ``` - -1. Run the `chef-client` command on the node using the `knife ssh` utility. This command causes the node to pull the recipes in its run list from the server. It also determines whether there are any updates. The Chef Server transmits the recipes to the target node. When the recipe runs, it deletes the file and installs a cron job to keep the node up to date in the future. In the following command, replace `nodename` with the actual name of the target node. Replace `username` with the name of a user account with `sudo` access. Enter the password for the account when prompted to do so. - - ```command - knife ssh 'name:nodename' 'sudo chef-client' -x username - ``` - - ```output - nodename Chef Infra Client, version 17.10.3 - nodename Patents: https://www.chef.io/patents - nodename Infra Phase starting - nodename Resolving cookbooks for run list: ["cron-delvalidate::default"] - nodename Synchronizing cookbooks: - nodename - cron-delvalidate (0.1.3) - nodename Installing cookbook gem dependencies: - nodename Compiling cookbooks... - nodename Loading Chef InSpec profile files: - nodename Loading Chef InSpec input files: - nodename Loading Chef InSpec waiver files: - nodename Converging 2 resources - nodename Recipe: cron-delvalidate::default - nodename * cron[clientrun] action create - nodename - add crontab entry for cron[clientrun] - nodename * file[/etc/chef/validation.pem] action delete (up to date) - nodename - nodename Running handlers: - nodename Running handlers complete - nodename Infra Phase complete, 1/2 resources updated in 03 seconds - ``` - -## Conclusion - -Chef is an infrastructure as code (IaC) application for automating the deployment and management of infrastructure nodes. The Chef architecture consists of the Chef Server, which stores all the procedures, and a Chef Workstation, where the infrastructure code is developed. The managed nodes communicate with the server to receive updates. To use Chef, install the Chef Server and Chef Workstation software. Share RSA keys between the server and workstation, and install version control and the Chef Knife utility on the workstation. Bootstrap the target nodes using the `knife bootstrap` utility. After a node is bootstrapped, it is possible to download cookbooks and recipes using the node's run list. For more information, see the [Chef documentation](https://docs.chef.io/). \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/puppet/install-and-manage-mysql-databases-with-puppet-hiera-on-ubuntu-18-04/index.md b/docs/guides/applications/configuration-management/puppet/install-and-manage-mysql-databases-with-puppet-hiera-on-ubuntu-18-04/index.md deleted file mode 100644 index b9f737d42be..00000000000 --- a/docs/guides/applications/configuration-management/puppet/install-and-manage-mysql-databases-with-puppet-hiera-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,241 +0,0 @@ ---- -slug: install-and-manage-mysql-databases-with-puppet-hiera-on-ubuntu-18-04 -title: "Manage MySQL Database with Puppet Hiera on Ubuntu 18.04" -description: "Use this guide to install Puppet with MySQL modules and Puppet Hiera configuration manifests to manage MySQL in a variety of environments on Ubuntu 18.04." -authors: ["Linode"] -contributors: ["Linode"] -published: 2019-01-15 -keywords: ["puppet installation", "configuration change management", "server automation", "mysql", "database", "hiera"] -tags: ["database","ubuntu","automation","mysql"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Puppet Labs](https://puppet.com/)' - - '[Puppet Open Source Documentation](https://puppet.com/docs/open-source-puppet/)' - - '[The Puppet Forge](https://forge.puppet.com/)' - - '[Hiera documentation](https://docs.puppet.com/hiera/)' - - '[Facter](https://puppet.com/docs/puppet/7/facter.html)' -relations: - platform: - key: install-puppet-mysql-hiera - keywords: - - distribution: Ubuntu 18.04 -aliases: [] ---- - -[Puppet](https://puppet.com/) is a configuration management system that helps simplify the use and deployment of different types of software, making system administration more reliable and replicable. In this guide, we use Puppet to manage an installation of [MySQL](https://www.mysql.com/), a popular relational database used for applications such as WordPress, Ruby on Rails, and others. [Hiera](https://docs.puppet.com/hiera/) is a method of defining configuration values that Puppet will use to simplify MySQL configuration. - -In this guide, you'll use Puppet to deploy [modules](https://docs.puppet.com/puppet/latest/modules_fundamentals.html) on your server. At the end, you will have MySQL installed, configured, and ready to use for a variety of applications that require a database backend. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -## Install and Configure Puppet - -Follow these steps to set up Puppet for single-host, local-only deployment. If you need to configure more than one server or to deploy a Puppet master, follow our [multi-server Puppet guide](/cloud/guides/install-and-configure-puppet). - -### Install the Puppet Package - -1. Install the `puppetlabs-release-bionic` repository to add the Puppet packages: - - wget https://apt.puppet.com/puppet-release-bionic.deb - sudo dpkg -i puppet-release-bionic.deb - -2. Update the apt package index to make the Puppet Labs repository packages available, then install Puppet. This will install the `puppet-agent` package, which provides the `puppet` executable within in a compatible Ruby environment: - - sudo apt update && sudo apt install puppet-agent - -3. Confirm the version of Puppet installed: - - puppet --version - - At the time of writing, the Puppet version is `6.1.0`. - -### Install the Puppet MySQL Module - -[Puppet Forge](https://forge.puppet.com/) is a collection of _modules_ that aid in the installation of different types of software. The [MySQL module](https://forge.puppet.com/puppetlabs/mysql) handles the installation and configuration of MySQL without you needing to manage various configuration files and services by hand. - -1. Install the MySQL module: - - sudo puppet module install puppetlabs-mysql --version 7.0.0 - - This will install the `mysql` module into the default path: `/etc/puppetlabs/code/environments/production/modules/`. - -### Puppet MySQL Manifest - -This guide uses a Puppet *manifest* to provide Puppet with installation and configuration instructions. Alternatively, you can configure [a Puppet master](/cloud/guides/install-and-configure-puppet). - -While the entirety of a Puppet *manifest* can contain the desired configuration for a host, values for Puppet *classes* or *types* can also be defined in a Hiera configuration file to simplify writing Puppet manifests in most cases. In this example, the `mysql::server` class parameters will be defined in Hiera, but the class must first be applied to the host. - -To apply the `mysql::server` class to all hosts by default, create the following Puppet manifest: - -{{< file "/etc/puppetlabs/code/environments/production/manifests/site.pp" puppet >}} -include ::mysql::server -{{< /file >}} - - -Note that `site.pp` is the default manifest file. Without a qualifying `node { .. }` line, this applies the class to any host applying the manifest. Puppet now knows to apply the `mysql::server` class, but still needs values for resources like databases, users, and other settings. Configure Hiera to provide these values in the next section. - -## Install and Configure Puppet Hiera - -To understand how Hiera works, consider this excerpt from the default `hiera.yaml` file: - -{{< file "/etc/puppetlabs/code/environments/production/hiera.yaml" yaml >}} ---- -version: 5 -hierarchy: - - name: "Per-node data" - path: "nodes/%{::trusted.certname}.yaml" - - name: "Common data" - path: "common.yaml" -{{< /file >}} - - -This Hiera configuration instructs Puppet to accept variable values from `nodes/%{::trusted.certname}.yaml`. If your Linode's hostname is `examplehostname`, define a file called `nodes/examplehostname.yaml`). Any variables found in YAML files higher in the hierarchy are preferred, while any variable names that do not exist in those files will fall-through to files lower in the hierarchy (in this example, `common.yaml`). - -The following configuration will define Puppet variables in `common.yaml` to inject variables into the `mysql::server` class. - -### Initial Hiera Configuration - -Hiera configuration files are formatted as yaml, with keys defining the Puppet parameters to inject their associated values. To get started, set the MySQL root password. The following example of a Puppet manifest is one way to control this password: - -{{< file "example.pp" >}} -class { '::mysql::server': - root_password => 'examplepassword', -} -{{< /file >}} - -We can also define the root password with the following Hiera configuration file. Create the following YAML file and note how the `root_password` parameter is defined as Hiera yaml: - -{{< file "/etc/puppetlabs/code/environments/production/data/common.yaml" >}} -mysql::server::root_password: examplepassword -{{< /file >}} - -Replace `examplepassword` with the secure password of your choice. Run Puppet to set up MySQL with default settings and the chosen root password: - - sudo -i puppet apply /etc/puppetlabs/code/environments/production/manifests/site.pp - -Puppet will output its progress before completing. To confirm MySQL has been configured properly, run a command: - - mysql -u root -p -e 'select version();' - -Enter the password and MySQL returns its version: - - +-------------------------+ - | version() | - +-------------------------+ - | 5.7.24-0ubuntu0.18.04.1 | - +-------------------------+ - -### Define MySQL Resources - -Using Hiera, we can define the rest of the MySQL configuration entirely in yaml. The following steps will create a database and user for use in a WordPress installation. - -1. Create a pre-hashed MySQL password. Replace the password `wordpresspassword` in this example, and when prompted for a the root MySQL password, use the first root password chosen in the previous section to authenticate. Note the string starting with a `*` that the command returns for Step 2: - - mysql -u root -p -NBe 'select password("wordpresspassword")' - *E62D3F829F44A91CC231C76347712772B3B9DABC - -2. With the MySQL password hash ready, we can define Hiera values. The following YAML defines parameters to create a database called `wordpress` and a user named `wpuser` that has permission to connect from `localhost`. The YAML also defines a `GRANT` allowing `wpuser` to operate on the `wordpress` database with `ALL` permissions: - - {{< file "/etc/puppetlabs/code/environments/production/data/common.yaml" yaml >}} -mysql::server::root_password: examplepassword -mysql::server::databases: - wordpress: - ensure: present -mysql::server::users: - wpuser@localhost: - ensure: present - password_hash: '*E62D3F829F44A91CC231C76347712772B3B9DABC' -mysql::server::grants: - wpuser@localhost/wordpress.*: - ensure: present - privileges: ALL - table: wordpress.* - user: wpuser@localhost - -{{< /file >}} - - -1. Re-run Puppet: - - sudo -i puppet apply /etc/puppetlabs/code/environments/production/manifests/site.pp - -2. The `wpuser` should now be able to connect to the `wordpress` database. To verify, connect to the MySQL daemon as the user `wpuser` to the `wordpress` database: - - mysql -u wpuser -p wordpress - - After you enter the password for `wpuser`, exit the MySQL prompt: - - exit - -### Add Hierarchies for Specific Environments - -Additional configurations can be added that will only be applied to specific environments. For example, backup jobs may only be applied for hosts in a certain region, or specific databases can be created in a particular deployment. - -In the following example, Puppet will configure the MySQL server with one additional database, but only if that server's distribution is Debian-based. - -1. Modify `hiera.yaml` to contain the following: - - {{< file "/etc/puppetlabs/code/environments/production/hiera.yaml" yaml >}} ---- -version: 5 -hierarchy: - - name: "Per OS Family" - path: "os/%{facts.os.family}.yaml" - - name: "Other YAML hierarchy levels" - paths: - - "common.yaml" -{{< /file >}} - - - This change instructs Hiera to look for Puppet parameters first in `"os/%{facts.os.family}.yaml"` and then in `common.yaml`. The first, fact-based element of the hierarchy is dynamic, and dependent upon the host that Puppet and Hiera control. In this Ubuntu-based example, Hiera will look for `Debian.yaml` in the `os` folder, while on a distribution such as CentOS, the file `RedHat.yaml` will automatically be referenced instead. - -1. Create the following YAML file: - - {{< file "/etc/puppetlabs/code/environments/production/data/os/Debian.yaml" yaml >}} -lookup_options: - mysql::server::databases: - merge: deep - -mysql::server::databases: - ubuntu-backup: - ensure: present - -{{< /file >}} - - - Though similar to the `common.yaml` file defined in previous steps, this file will add the `ubuntu-backup` database *only* on Debian-based hosts (like Ubuntu). In addition, the `lookup_options` setting ensures that the `mysql::server:databases` parameter is *merged* between `Debian.yaml` and `common.yaml` so that all databases are managed. Without `lookup_options` set to deeply merge these hashes, only the most specific hierarchy file will be applied to the host, in this case, `Debian.yaml`. - - * Alternatively, because our Puppet manifest is short, we can test the same command using the `-e` flag to apply an inline manifest: - - sudo -i puppet apply -e 'include ::mysql::server' - -3. Run Puppet and observe the changes: - - sudo -i puppet apply /etc/puppetlabs/code/environments/production/manifests/site.pp - -4. Verify that the new database exists: - - mysql -u root -p -e 'show databases;' - - This includes the new `ubuntu-backup` database: - - +---------------------+ - | Database | - +---------------------+ - | information_schema | - | mysql | - | performance_schema | - | sys | - | ubuntu-backup | - | wordpress | - +---------------------+ - -Congratulations! You can now control your Puppet configuration via highly configurable Hiera definitions. diff --git a/docs/guides/applications/configuration-management/salt/configure-apache-with-salt-stack/index.md b/docs/guides/applications/configuration-management/salt/configure-apache-with-salt-stack/index.md deleted file mode 100644 index a33c7c8db11..00000000000 --- a/docs/guides/applications/configuration-management/salt/configure-apache-with-salt-stack/index.md +++ /dev/null @@ -1,614 +0,0 @@ ---- -slug: configure-apache-with-salt-stack -title: "Configure Apache with Salt Stack" -description: 'This guide provides you with step-by-step instructions for installing and configuring the Apache Web Server on Ubuntu, Debian, and CentOS with the Salt Sack.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2018-10-19 -modified: 2019-01-02 -keywords: ['salt','stack','saltstack','apache','httpd','ubuntu','debian','centos'] -tags: ["automation","salt","debian","centos","ubuntu","apache"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: ConfigureApachewithSaltStack.png -external_resources: -- '[Salt Apache State Module](https://docs.saltstack.com/en/latest/ref/states/all/salt.states.apache.html)' -- '[Salt Apache_Conf State Module](https://docs.saltstack.com/en/latest/ref/states/all/salt.states.apache_conf.html)' -- '[Salt Apache_Site State Module](https://docs.saltstack.com/en/latest/ref/states/all/salt.states.apache_site.html)' -- '[Using Grains in SLS Modules](https://docs.saltstack.com/en/latest/topics/tutorials/states_pt3.html#using-grains-in-sls-modules)' -aliases: [] ---- - -Salt is a powerful configuration management tool. In this guide you will create Salt state files that are capable of installing and configuring Apache on Ubuntu 18.04, Debian 9, or CentOS 7. - -## Before You Begin - -You will need at least two Linodes with Salt installed. If you have not already, read our [Getting Started with Salt - Basic Installation and Setup Guide](/cloud/guides/getting-started-with-salt-basic-installation-and-setup) and follow the instructions for setting up a Salt master and minion. - -The following steps will be performed on your Salt master. - -{{< note >}} -The steps in this guide require root privileges. Be sure to run the steps below as `root` or with the `sudo` prefix. For more information on privileges, see our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Setting Up Your Salt Master and Managed Files - -### Salt Master SLS Files - -1. Create the `/srv/salt` directory if it does not already exist: - - mkdir /srv/salt - -1. Create a Salt top file in `/srv/salt` that will be Salt's entry point to the Apache configuration: - - {{< file "/srv/salt/top.sls" yaml >}} -base: - 'G@os_family:Debian': - - match: compound - - apache-debian - - 'G@os:CentOS': - - match: compound - - apache-centos -{{< /file >}} - - This top file uses [compound matching](https://docs.saltstack.com/en/latest/topics/targeting/compound.html) to target your minions by operating system using Salt Grains. This will allow Salt to choose the appropriate Apache configuration depending on the Linux distribution. These matchers could be extended to be even more specific. For instance, if you wanted to only target minions with the ID of `web-server` that are running on Ubuntu, you can type `web* and G@os:Ubuntu`. - -### Pillar Files - -1. Create the `/srv/pillar` directory if it does not already exist: - - mkdir /srv/pillar - -1. Create a Pillar top file. This top file references the `apache.sls` Pillar file that you will create in the next step: - - {{< file "/srv/pillar/top.sls" yaml >}} -base: - '*': - - apache -{{< /file >}} - -1. Create the `apache.sls` file that was referenced in the previous step. This file defines Pillar data that will be used inside our Apache state file in the next section, in this case your domain name. Replace `example.com` with your domain: - - {{< file "/srv/pillar/apache.sls" yaml >}} -domain: example.com -{{< /file >}} - -### Website Files - -1. Create a directory for your website files in the `/srv/salt` directory. Replace `example.com` with your website domain name: - - mkdir /srv/salt/example.com - - This directory will be accessible from your Salt state files at `salt://example.com`. - -1. Create an `index.html` file for your website in the `/srv/salt/example.com` directory, substituting `example.com` for the folder name you chose in the previous step. You will use this file as a test to make sure your website is functioning correctly. - - {{< file "/srv/salt/example.com/index.html" html >}} - - -

Server Up and Running!

- - -{{< /file >}} - -### Configuration Files - -1. Create a folder for your additional configuration files at `/srv/salt/files`. These files will be accessible at `salt://files`. - - mkdir /srv/salt/files - -1. Create a file called `tune_apache.conf` in `/srv/salt/files` and paste in the following block: - - {{< file "/srv/salt/files/tune_apache.conf" ApacheConf >}} - -StartServers 4 -MinSpareServers 20 -MaxSpareServers 40 -MaxClients 200 -MaxRequestsPerChild 4500 - -{{}} - - This MPM prefork module provides additional [tuning for your Apache installation](/cloud/guides/tuning-your-apache-server). This file will be managed by Salt and installed into the appropriate configuration directory in a later step. - -1. If you will be installing Apache on a CentOS machine, create a file called `include_sites_enabled.conf` in `/srv/salt/files` and paste in the following: - - {{< file "/srv/salt/files/include_sites_enabled.conf" ApacheConf >}} -IncludeOptional sites-enabled/*.conf -{{< /file >}} - - This file will allow us to use file directories like those found on Debian installations to help organize the Apache configuration. - -## Creating the Apache State File for Debian and Ubuntu - -### Individual Steps - -This guide will be going through the process of creating the Apache for Debian and Ubuntu state file step by step. If you would like to view the entirety of the state file, [you can view it at the end of this section](/cloud/guides/configure-apache-with-salt-stack#complete-state-file). - -1. Create a state file named `apache-debian.sls` in `/srv/salt` and open it in a text editor of your choice. - -1. Instruct Salt to install the `apache2` package and start the `apache2` service: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -apache2: - pkg.installed - -apache2 Service: - service.running: - - name: apache2 - - enable: True - - require: - - pkg: apache2 - -... -{{< /file >}} - - Here Salt makes sure the `apache2` package is installed with `pkg.installed`. Likewise, it ensures the `apache2` service is running and enabled under `service.running`. Also under `service.running`, `apache-debian.sls` uses `require` to ensure that this command does not run before the `apache2` package is installed. This `require` step will be repeated throughout `apache-debian.sls`. - - Lastly, a `watch` statement is employed to restart the `apache2` service if your site's configuration file changes. You will define that configuration file in a later step. Note that this configuration file is named using the domain you supplied when creating your Salt Pillar file in the first section. This Pillar data will be used throughout `apache-debian.sls`. - -1. Turn off KeepAlive: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -... - -Turn Off KeepAlive: - file.replace: - - name: /etc/apache2/apache2.conf - - pattern: 'KeepAlive On' - - repl: 'KeepAlive Off' - - show_changes: True - - require: - - pkg: apache2 -... -{{< /file >}} - - KeepAlive allows multiple requests to be sent over the same TCP connection. For the purpose of this guide KeepAlive will be disabled. To disable it, Salt is instructed to find the KeepAlive directive in `/etc/apache2/apache2.conf` by matching a pattern and replacing it with `KeepAlive Off`. `show_changes` instructs Salt to display any changes it has made during a highstate. - -1. Transfer `tune_apache.conf` to your minion and enable it: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -... - -/etc/apache2/conf-available/tune_apache.conf: - file.managed: - - source: salt://files/tune_apache.conf - - require: - - pkg: apache2 - -Enable tune_apache: - apache_conf.enabled: - - name: tune_apache - - require: - - pkg: apache2 - -... -{{< /file >}} - - This step takes the `tune_apache.conf` file you created in the [Configuration Files](/cloud/guides/configure-apache-with-salt-stack#configuration-files) step and transfers it to your Salt minion. Then, Salt enables that configuration file with the [apache_conf module](https://docs.saltstack.com/en/latest/ref/states/all/salt.states.apache_conf.html). - -1. Create the necessary directories: - - {{< file "/srv/salt/apache-debian.sls" yaml>}} -... - -/var/www/html/{{ pillar['domain'] }}: - file.directory - -/var/www/html/{{ pillar['domain'] }}/log: - file.directory - -/var/www/html/{{ pillar['domain'] }}/backups: - file.directory - -/var/www/html/{{ pillar['domain'] }}/public_html: - file.directory - -... -{{< /file >}} - -1. Disable the default virtual host configuration file: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -... - -000-default: - apache_site.disabled: - - require: - - pkg: apache2 - -... -{{< /file >}} - - This step uses Salt's [apache_site module](https://docs.saltstack.com/en/latest/ref/states/all/salt.states.apache_site.html) to disable the default Apache virtual host configuration file, and is the same as running `a2dissite` on a Debian-based machine. - -1. Create your site's virtual host configuration file: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -... - -/etc/apache2/sites-available/{{ pillar['domain'] }}.conf: - apache.configfile: - - config: - - VirtualHost: - this: '*:80' - ServerName: - - {{ pillar['domain'] }} - ServerAlias: - - www.{{ pillar['domain'] }} - DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html - ErrorLog: /var/www/html/{{ pillar['domain'] }}/log/error.log - CustomLog: /var/www/html/{{ pillar['domain'] }}/log/access.log combined - -... -{{< /file >}} - - This step uses Salt's [apache module](https://docs.saltstack.com/en/latest/ref/states/all/salt.states.apache.html), (not to be confused with the `apache_site` module used in the previous step), to create your site's virtual host configuration file. The `this` variable signifies what would traditionally be include with `VirtualHost` within angle brackets in an Apache configuration file: ``. - -1. Enable your new virtual host configuration file: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -... - -{{ pillar['domain'] }}: - apache_site.enabled: - - require: - - pkg: apache2 - -... -{{< /file >}} - - This step uses the same `apache_site` module you used to disable the default virtual host file to enable your newly created virtual host file. `apache_site.enabled` creates a symlink from `/etc/apache2/sites-available/example.com.conf` to `/etc/apache2/sites-enabled/example.com.conf` and is the same as running `a2ensite` on a Debian-based machine. - -1. Transfer your `index.html` website file to your minion: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -... - -/var/www/html/{{ pillar['domain'] }}/public_html/index.html: - file.managed: - - source: salt://{{ pillar['domain'] }}/index.html -{{< /file >}} - - Any changes made to your `index.html` file on your Salt master will be propagated to your minion. - - {{< note respectIndent=false >}} -Since Salt is not watching configuration files for a change to trigger a restart for Apache, you may need to use the command below from your Salt master. - - salt '*' apache.signal restart -{{< /note >}} - -### Complete State File -The complete `apache-debian.sls` file looks like this: -{{< file "/srv/salt/apache-debian.sls" yaml >}} -apache2: - pkg.installed - -apache2 Service: - service.running: - - name: apache2 - - enable: True - - require: - - pkg: apache2 - -Turn Off KeepAlive: - file.replace: - - name: /etc/apache2/apache2.conf - - pattern: 'KeepAlive On' - - repl: 'KeepAlive Off' - - show_changes: True - - require: - - pkg: apache2 - -/etc/apache2/conf-available/tune_apache.conf: - file.managed: - - source: salt://files/tune_apache.conf - - require: - - pkg: apache2 - -Enable tune_apache: - apache_conf.enabled: - - name: tune_apache - - require: - - pkg: apache2 - -/var/www/html/{{ pillar['domain'] }}: - file.directory - -/var/www/html/{{ pillar['domain'] }}/log: - file.directory - -/var/www/html/{{ pillar['domain'] }}/backups: - file.directory - -/var/www/html/{{ pillar['domain'] }}/public_html: - file.directory - -000-default: - apache_site.disabled: - - require: - - pkg: apache2 - -/etc/apache2/sites-available/{{ pillar['domain'] }}.conf: - apache.configfile: - - config: - - VirtualHost: - this: '*:80' - ServerName: - - {{ pillar['domain'] }} - ServerAlias: - - www.{{ pillar['domain'] }} - DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html - ErrorLog: /var/www/html/{{ pillar['domain'] }}/log/error.log - CustomLog: /var/www/html/{{ pillar['domain'] }}/log/access.log combined - -{{ pillar['domain'] }}: - apache_site.enabled: - - require: - - pkg: apache2 - -/var/www/html/{{ pillar['domain'] }}/public_html/index.html: - file.managed: - - source: salt://{{ pillar['domain'] }}/index.html -{{< /file >}} - -## Creating an Apache State File for CentOS - -### Individual Steps - -1. Create a file called `apache-centos.sls` in `/srv/salt` and open it in a text editor of your choice. - -2. On CentOS Apache is named `httpd`. Instruct Salt to install `httpd` and run the `httpd` service: - - {{< file "/srv/salt/apache-centos.sls" yaml>}} -httpd: - pkg.installed - -httpd Service: - service.running: - - name: httpd - - enable: True - - require: - - pkg: httpd - - watch: - - file: /etc/httpd/sites-available/{{ pillar['domain'] }}.conf - -... -{{< /file >}} - - Here Salt makes sure the `httpd` package is installed with `pkg.installed`. Likewise, it ensures the `httpd` service is running and enabled under `service.running`. Also under `service.running`, `apache-debian.sls` uses `require` to ensure that this command does not run before the `httpd` package is installed. This `require` step will be repeated throughout `apache-centos.sls`. - - Lastly, a `watch` statement is employed to restart the `httpd` service if your site’s configuration file changes. You will define that configuration file in a later step. Note that this configuration file is named using the domain you supplied when creating your Salt Pillar file in the first section. This Pillar data will be used throughout `apache-centos.sls`. - -1. Turn off KeepAlive: - - {{< file "/srv/salt/apache-centos.sls" yaml >}} -... - -Turn Off KeepAlive: - file.replace: - - name: /etc/httpd/conf/httpd.conf - - pattern: 'KeepAlive On' - - repl: 'KeepAlive Off' - - show_changes: True - - require: - - pkg: httpd -... -{{< /file >}} - - KeepAlive allows multiple requests to be sent over the same TCP connection. For the purpose of this guide KeepAlive will be disabled. To disable it, Salt is instructed to find the KeepAlive directive in `/etc/httpd/conf/httpd.conf` by matching a pattern and replacing it with `KeepAlive Off`. `show_changes` instructs Salt to display any changes it has made during a highstate. - -1. Change the DocumentRoot: - - {{< file "/srv/salt/apache-centos.sls" yaml >}} -... - -Change DocumentRoot: - file.replace: - - name: /etc/httpd/conf/httpd.conf - - pattern: 'DocumentRoot "/var/www/html"' - - repl: 'DocumentRoot "/var/www/html/{{ pillar['domain'] }}/public_html"' - - show_changes: True - - require: - - pkg: httpd - -... -{{< /file >}} - - Similar to the last step, in this step `salt-centos.sls` instructs Salt to search for the DocumentRoot directive in Apache's `httpd.conf` file, and replaces that line with the new document root. This allows for the use of a Debian-style site directory architecture. - -1. Transfer the `tune_apache.conf` and `include_sites_enabled.conf` to your minion. - - {{< file "/srv/salt/apache-centos.sls" yaml>}} -... - -/etc/httpd/conf.d/tune_apache.conf: - file.managed: - - source: salt://files/tune_apache.conf - - require: - - pkg: httpd - -/etc/httpd/conf.d/include_sites_enabled.conf: - file.managed: - - source: salt://files/include_sites_enabled.conf - - require: - - pkg: httpd - -... -{{< /file >}} - -1. Create the necessary directories: - - {{< file "srv/salt/apache-centos.sls" yaml >}} -... - -/etc/httpd/sites-available: - file.directory - -/etc/httpd/sites-enabled: - file.directory - -/var/www/html/{{ pillar['domain'] }}: - file.directory - -/var/www/html/{{ pillar['domain'] }}/backups: - file.directory - -/var/www/html/{{ pillar['domain'] }}/public_html: - file.directory - -... -{{< /file >}} - -1. Create your site's virtual host configuration file: - - {{< file "/srv/salt/apache-centos.sls" yaml>}} -... - -/etc/httpd/sites-available/{{ pillar['domain'] }}.conf: - apache.configfile: - - config: - - VirtualHost: - this: '*:80' - ServerName: - - {{ pillar['domain'] }} - ServerAlias: - - www.{{ pillar['domain'] }} - DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html - file.symlink: - - target: /etc/httpd/sites-enabled/{{ pillar['domain'] }}.conf - - force: True - -... -{{< /file >}} - - This step uses Salt's [apache module](https://docs.saltstack.com/en/latest/ref/states/all/salt.states.apache.html) to create your site's virtual host configuration file. The `this` variable signifies what would traditionally be include with `VirtualHost` within angle brackets in an Apache configuration file: ``. - -1. Transfer your `index.html` website file to your minion: - - {{< file "/srv/salt/apache-debian.sls" yaml >}} -... - -/var/www/html/{{ pillar['domain'] }}/public_html/index.html: - file.managed: - - source: salt://{{ pillar['domain'] }}/index.html - -... -{{< /file >}} - - Any changes made to your `index.html` file on your Salt master will be propigated to your minion. - -1. Configure your firewall to allow http and https traffic: - - {{< file "/srv/salt/apache-centos.sls" yaml >}} -... - -Configure Firewall: - firewalld.present: - - name: public - - ports: - - 22/tcp - - 80/tcp - - 443/tcp -{{< /file >}} - - {{< note respectIndent=false >}} -It is imperative that you list all ports you need open to your machine in this section. Failure to list these ports will result in their closure by Salt. -{{< /note >}} - -### Complete State File - -The complete `apache-centos.sls` file looks like this: - - {{< file "/srv/salt/apache-centos.sls" yaml >}} -httpd: - pkg.installed - -httpd Service: - service.running: - - name: httpd - - enable: True - - require: - - pkg: httpd - - watch: - - file: /etc/httpd/sites-available/{{ pillar['domain'] }}.conf - -Turn off KeepAlive: - file.replace: - - name: /etc/httpd/conf/httpd.conf - - pattern: 'KeepAlive On' - - repl: 'KeepAlive Off' - - show_changes: True - - require: - - pkg: httpd - -Change DocumentRoot: - file.replace: - - name: /etc/httpd/conf/httpd.conf - - pattern: 'DocumentRoot "/var/www/html"' - - repl: 'DocumentRoot "/var/www/html/{{ pillar['domain'] }}/public_html"' - - show_changes: True - - require: - - pkg: httpd - -/etc/httpd/conf.d/tune_apache.conf: - file.managed: - - source: salt://files/tune_apache.conf - - require: - - pkg: httpd - -/etc/httpd/conf.d/include_sites_enabled.conf: - file.managed: - - source: salt://files/include_sites_enabled.conf - - require: - - pkg: httpd - -/etc/httpd/sites-available: - file.directory - -/etc/httpd/sites-enabled: - file.directory - -/var/www/html/{{ pillar['domain'] }}: - file.directory - -/var/www/html/{{ pillar['domain'] }}/backups: - file.directory - -/var/www/html/{{ pillar['domain'] }}/public_html: - file.directory - -/etc/httpd/sites-available/{{ pillar['domain'] }}.conf: - apache.configfile: - - config: - - VirtualHost: - this: '*:80' - ServerName: - - {{ pillar['domain'] }} - ServerAlias: - - www.{{ pillar['domain'] }} - DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html - file.symlink: - - target: /etc/httpd/sites-enabled/{{ pillar['domain'] }}.conf - - force: True - -/var/www/html/{{ pillar['domain'] }}/public_html/index.html: - file.managed: - - source: salt://{{ pillar['domain'] }}/index.html - -Configure Firewall: - firewalld.present: - - name: public - - ports: - - 22/tcp - - 80/tcp - - 443/tcp -{{< /file >}} - -## Running the Apache State File - -On your Salt master, issue a highstate command: - - salt '*' state.apply - -After a few moments you should see a list of Salt commands and a summary of their successes. Navigate to your website's domain name if you have your DNS set up already, or your website's public IP address. You should see your `index.html` file. You have now used Salt to configure Apache. Visit the links in the section below for more information. diff --git a/docs/guides/applications/containers/serverless-vs-containers/index.md b/docs/guides/applications/containers/serverless-vs-containers/index.md deleted file mode 100644 index 00b3dc9e469..00000000000 --- a/docs/guides/applications/containers/serverless-vs-containers/index.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -slug: serverless-vs-containers -title: "Serverless vs Containers: Choose Which One to Use" -description: 'What are the similarities and differences between serverless and containers? What should developers consider when choosing one?' -authors: ["John Mueller"] -contributors: ["John Mueller"] -published: 2023-05-02 -keywords: ['serverless vs. containers','serverless computing','serverless applications','containers','microservices','serverless architecture','serverless web applications','backend services'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -Getting as much done in as little time, and with as little overhead as possible is an essential element of modern development. The end result needs to be easy to deploy, maintain, and debug. Plus, it all has to run in the cloud. That’s asking a lot, but both serverless applications and containers offer solutions. They’re both designed to replace virtual machines that require much more work on the organization’s part. Of the two, containers are a more robust solution that include everything needed to run an application, usually as a set of microservices. Meanwhile, serverless applications are less complex solutions that focus on application code that relies on services provided by a vendor API. There isn’t a right or wrong choice with either technology, simply the choice that benefits an organization most, as this guide explains. - -## What Is Serverless? - -Serverless applications are hosted on a vendor system, where the functions used to create them respond to events. With *serverless*, the developer focuses on the code, the application, and not the server or hardware. The infrastructure, such as backend services and libraries, are generally provided by the vendor. Therefore the developer is only looking at the application code, and not its dependencies. - -A serverless application provides functionality like automatic scaling, provisioning, built-in service integration, automated configuration, and high-availability. All without any additional effort on the developer’s part. Because of the manner of hosting, serverless applications can save an organization a huge amount of money on dependency costs. - -A serverless application can provide traditional desktop support, backend services, and serverless web applications. When compared to a microservice, the serverless application represents a method of running an application, while a microservice represents a method of designing an application. Additionally, serverless applications, unlike microservices, don’t run continuously, do require an event to begin execution, and individual functions perform exactly one task. A microservice can run continuously over a long period of time, and can support more than one task or function. The advantage of using a serverless application over a microservice is that the serverless application waits for an event, runs, and then stops. The costs of running a serverless application is therefore less than a microservice in situations where an application is expected to receive frequent usage spikes. - -### What is Serverless Used For? - -Serverless applications are perfect for mobile and web application startups because of their low startup costs and ability to handle lightweight applications. They are often employed in these use cases: - -- Situations where the traffic is unpredictable -- Internet of Things (IoT) applications -- Any application that sees constant and significant changes -- Applications where it’s possible to break tasks down into single functions and then compose these functions together to create a Packaged Business Capability (PBC) - -### Considering the Serverless Application Process - -As with most application development, there is a process to follow in order to architect a serverless application. It doesn’t matter whether the resulting software represents backend services, frontend services, or both. This process is inherently different from working with monolithic applications, microservices, [Packaged Business Capabilities (PBCs)](https://www.elasticpath.com/blog/what-is-the-difference-between-PBCs-and-microservices), container applications, or any number of other software development patterns. The idea is to break a software requirement down into smaller pieces until it’s possible to describe a particular piece very simply. Here’s how: - -1. Define individual services that perform a specific task. -1. Define individual functions (elements that perform one and only one task) to make up the services. -1. Define events that fire the functions, remembering that serverless applications work around the idea that a function starts, performs a task, and then stops. -1. Create configuration files that describe each function. -1. Create a configuration provider file that describes how the function is to interact with the framework that supports the serverless application. -1. Create a service configuration that describes the provider file, function files, and any plugins that make up the service. - -## What Are Containers? - -Containers differ from serverless applications because a container has everything needed to run the application, such as libraries, system settings, and other dependencies. This additional content over a serverless application means that the developer needs to be concerned about the application code and everything that goes with it. Consequently, there is more work for the developer. However, containers have some serious benefits over serverless applications, one of which is a lack of vendor lock-in. For instance, a [Docker](https://www.docker.com/) container application can run on any system that supports Docker. Just like containers used for shipping, container applications are standardized. They can be moved anywhere, on any system, without regard to the underlying hardware, or operating system details. - -A container is focused on just one application, unlike a virtual machine, which imitates an entire computer, operating system, and all. A container is simpler and less resource intensive. Given an application of equal complexity, it’s possible to run more containers on a physical piece of hardware than virtual machines. On the other hand, a virtual machine can run multiple applications. A major difference between containers and virtual machines is that containers share a single kernel (operating system) on a physical machine. Meanwhile, virtual machines each have their own kernel. Consequently, all container applications running on a physical device must be compatible with the one kernel. Using a virtual machine offers the opportunity to use the particular kernel that works best with the applications in question. - -### What Are Containers Used For? - -Containers are often used for the following purposes: - -- Deploying API endpoints -- Deploying repetitive jobs and tasks -- Providing devops support for Continuous Integration and Continuous Deployment (CI/CD) -- Hosting background processing applications -- Handling event-driven processing -- Running microservices -- Moving large legacy applications to the cloud - -### Considering the Container Application Process - -As with serverless applications, there is a common process used to create container applications of all sorts. Generally, this process follows these steps: - -1. [Break an existing monolithic application down into microservices](https://martinfowler.com/articles/break-monolith-into-microservices.html) as necessary. -1. Create a new container image based on an existing image template. -1. Add code, resources, and other application files to the image using host commands. -1. Configure the image’s startup commands using host commands. -1. Build and run the image from within the container (rather than externally as normal). -1. Deploy the image using the host server’s instance service. - -## What Are Similarities Between Serverless and Containers? - -Serverless applications and containers embrace similar strategies of breaking solutions down into smaller, more manageable pieces. They also have the same goals of reducing costs, development time, and maintenance time, while creating a more flexible environment. - -## What Are the Key Differences Between Serverless and Containers? - -In addition to the differences already mentioned, it’s possible to compare them in specific ways. Most notably, the two technologies have differences in the manner they use physical machines, scale, keep costs low, and manage deployment details. - -### Physical Machines - -A serverless application can live on multiple physical machines, while container applications always reside on just one physical machine. The ability to live on multiple machines gives serverless applications a resource availability advantage without a lot of extra work on the developer’s part. However, techniques like load balancing can be used to divide the load between multiple instances of a container application on multiple physical systems. The end result is seemingly the same, but the container application requires more configuration and implementation. - -### Scalability - -Serverless applications have an advantage when it comes to scalability because they automatically scale. The hosting vendor provides as little or as much computing power as needed to handle a particular load at a given time. When working with container applications, a developer needs to allocate enough containers to handle the anticipated load. If the load exceeds expectations, the application begins to run slowly, negatively impacting customers. When the load is less than expected, an organization wastes money on unused resources. It’s entirely possible to find cloud providers that have automatic scaling at the virtual machine level. While this can help mitigate the container disadvantage to some degree, this is *configurable* by the developer, but not *managed* by them. - -### Cost - -Serverless applications run only when they need to, which means they cost less to operate than containers when viewed directly. However, a problem occurs when considering the cost of application latency. Because a container is always running, it provides an immediate response to any request. If a serverless application needs to be loaded from outside the cache, there is additional time to consider before the task completes. Time is money. Consequently, even for loads where the requests are consistent, container applications may actually cost less because they’re more responsive. - -### Deployment Time - -The time to deploy applications has consistently gotten shorter. What used to take months using physical systems and minutes using virtual machines, now takes seconds using containers, and milliseconds using serverless applications. A serverless application developer generally has a deployment time advantage because there are no underlying system dependencies to configure and serverless applications are smaller. - -### Maintenance - -Serverless applications require less direct maintenance than containers because the hosting service addresses all of the maintenance needs. In an ideal situation, this means a serverless application developer has a significant advantage in time because the container developer must address low-level maintenance. However, the serverless application scenario can also experience problems. For example, unexpected or unwanted updates pushed by a vendor who has an interest in keeping everything up-to-date. Because a container developer has direct control over the underlying details, maintenance can be performed at a time most beneficial to the container application. This potentially saves time in the long term. - -### Testing - -Application testing is challenging when working with a serverless application because of how it runs. An event triggers the function, which performs the task and immediately shuts down. A developer is often forced to use application logs to locate the source of a problem. Container applications run continuously and in the same manner no matter where they run. In this case, the developer often has standardized tools to use in the debugging process. Many IDEs, such as [IntelliJ IDEA](https://www.jetbrains.com/help/idea/debug-a-java-application-using-a-dockerfile.html), are set up to debug container applications. - -## What Factors Should Developers Use to Choose Which One to Use? - -Serverless applications offer reduced deployment time, fewer maintenance requirements, and cost less when working with a load that can suddenly spike. They’re an optimal choice for startups that have smaller, less complex applications to manage without special underlying support needs. - -Container applications offer reduced costs for consistent loads and a great deal more in application configuration flexibility. They’re an optimal choice when moving a legacy application from local servers to the cloud. - -## Conclusion - -Serverless applications and containers both have advantages and disadvantages. Sometimes the best option is not to make a choice, but rather to use the technology that fits a specific need. Parts of a solution can run as serverless applications and other parts can run as containers. Of course, this combined option has drawbacks, too. Not the least of which is having to manage two different technologies for a single solution. This increases complexity and potentially reduces both reliability and security. \ No newline at end of file diff --git a/docs/guides/applications/containers/what-are-cloud-native-applications/index.md b/docs/guides/applications/containers/what-are-cloud-native-applications/index.md deleted file mode 100644 index 561195856a1..00000000000 --- a/docs/guides/applications/containers/what-are-cloud-native-applications/index.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -slug: what-are-cloud-native-applications -title: "What Are Cloud-Native Applications" -description: 'Learn what cloud native means, and how cloud-native apps differ from traditional on-premises applications.' -authors: ["Andy Patrizio"] -contributors: ["Andy Patrizio"] -published: 2023-06-12 -keywords: ['cloud-native applications','cloud native apps','microservices','kubernetes','docker'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -"Cloud-native applications" is a term that has evolved as it has grown. As part of that growth, the [Cloud Native Computing Foundation](https://www.cncf.io/) (CNCF) was launched in 2015 by the Linux Foundation. It serves as a vendor-neutral platform for many projects around cloud-native apps, including Kubernetes, Prometheus, and Envoy. - -Cloud-native is an app modernization technique, usually driven by a business need to modernize applications as much as possible to drastically improve scale. Older, on-premises, monolithic applications are broken up into autonomous subsystems called microservices. These microservices can be deployed and scaled independently from other areas of the application. This allows the components to evolve independently as needed, and scale up more than would be possible in the old design. - -Considering the term is cloud *native*, it implies that apps live in the public cloud, as opposed to an on-premises data center. This is true, as they can be set up to run in cloud data centers owned by Amazon, Microsoft, or Google. However, they can also run in a private cloud within your own environment. - -The CNCF defines cloud-native computing an open source software stack used for containers. Each part of the app is packaged in its own container and dynamically orchestrated. They are actively scheduled and managed to optimize resource utilization. They are microservices-oriented to increase overall agility and maintainability. - -## Cloud vs. On-Premises - -In contrast to on-premises apps, cloud-native apps are architected specifically to run in the elastic and distributed nature of the cloud. One of the key differences between on-premises and the cloud is that cloud usage is metered. You pay for every CPU cycle, disk I/O, and even network traffic. - -On-premises apps have no such restrictions. It’s your environment, so apps are limited only by hardware resources and other apps on your network. - -Another key difference is on-premesis apps are usually monolithic, meaning all of the functions are contained in the app. Meanwhile, cloud-based apps are broken up into a microservices architecture. Here, functions are separated, so one specific function can be used without needing the rest. - -Because of this, cloud apps are described as "loosely coupled". This means that the code is not hardwired to any of the infrastructure components, so the app can scale up and down on demand. These architectures are typically built using microservices, but it's not mandatory. - -## What Are Microservices? - -Red Hat defines microservices as an architectural approach to building applications. As an architectural framework, microservices are distributed and loosely coupled, so one team’s changes won’t break the entire app. The benefit to using microservices is that development teams are able to rapidly build new components of apps to meet changing business needs. - -The microservice architecture enables the rapid, frequent, and reliable delivery of large, complex applications as needed. - -For example, when Microsoft issues its monthly fixes, does it send down a whole new installation of Windows? No, that would be completely inefficient when only a small piece of the overall operating system is being updated. Instead, the company pushes out a few megabytes of updated code to replace the existing code. - -With a monolithic on-prem app, updates mean that you have to update the whole app. With microservices, you just push the updated code. - -Cloud-native apps run in what is known as a container. Containers are similar to virtual machines, but slimmer. A virtual machine has a full operating system and gigabytes allocated to it, while containers have only a sliver of the OS and megabytes allocated. - -Containers effectively virtualize the host operating system (or kernel) and isolate an application’s dependencies from other containers running on the same machine. So if a container crashes, it doesn’t take down the whole app. - -The most popular container manager is Docker, however, there are a large [number of competitors](https://www.winosbite.com/docker-alternatives/). - -Kubernetes is another important component of cloud native. Developed by Google, Kubernetes is an open source container management platform that unifies a cluster of machines into a single pool of compute resources. Kubernetes organizes applications into groups of containers using the Docker engine, and keeps your application running as intended. - -## Key Differences Between Cloud-Native and On-Premises Applications - -Cloud-native requires a very different architecture than traditional on-premises enterprise applications. Here are the key differences: - -1. **Languages**: On-premises apps tend to be written in traditional languages like C/C++ and enterprise Java. If it’s on a mainframe, it’s likely in COBOL. Cloud-native apps are more likely to be written in a Web-centric language like Java, JavaScript, .Net, Node.js, PHP, Python, and Ruby. - -1. **Updateable**: Cloud-native apps are updated far more regularly and routinely through a DevOps process known as Constant Iteration and Constant Delivery (CI/CD). On-premises apps also require downtime as updates are installed, while cloud-native apps are always available. - -1. **Resilience**: Because the microservices architecture breaks an app into its core functions, each function is called a *service* that can be built and deployed independently. This means that individual services can function without negatively affecting others. If a function crashes, it doesn’t bring down the whole system like an on-premises app would. - -1. **Elasticity**: Cloud-native apps take advantage of the elasticity of the cloud by dynamically increasing resources when there is a spike in usage. If your cloud-based app experiences a burst in use, extra compute resources are automatically made available until the spike subsides and those extra resources are then deallocated. An on-premises app can’t do that. - -1. **Multi-Tenancy**: A cloud-native app has no problem working in a virtualized space and sharing resources with other apps. That’s what they are designed for. Most on-premises apps don’t work well in a virtual environment, or don’t work at all, thus requiring all the resources of a server. - -1. **Connected Resources**: An on-premises app usually has hard-coded connections to resources, such as networks, security, and storage. That means things can break if anything is moved or changed. A cloud-native app can automatically find those resources without requiring human intervention. - -1. **Automated**: Unlike on-premises apps, much of the cloud is automated, and that includes app management. Container management tools like Docker and Kubernetes automate everything, including scale up/scale down, self-service, rollback of problematic apps, and auditing of the app’s performance. With on-premises, this is usually done manually. - -## The Challenges - -Because cloud-native and on-premises are so different, the biggest mistake is to do what is called "lift and shift". This is where old on-premises apps are simply moved to AWS or Azure unchanged. This completely fails to take advantage of everything the cloud has to offer, such as scale and ease of updates. - -The real decision is whether to migrate an old on-premises app to the cloud, or simply rewrite from scratch. Here, the rule of thumb is that the more rewriting you have to do, the more appealing a rewrite from scratch becomes. \ No newline at end of file diff --git a/docs/guides/applications/media-servers/how-to-install-jellyfin/index.md b/docs/guides/applications/media-servers/how-to-install-jellyfin/index.md deleted file mode 100644 index 82a2282bb01..00000000000 --- a/docs/guides/applications/media-servers/how-to-install-jellyfin/index.md +++ /dev/null @@ -1,191 +0,0 @@ ---- -slug: how-to-install-jellyfin -title: How to Install Jellyfin on Linux -description: 'This guide will show you how to install Jellyfin, an open-source solution for organizing your media, on a Linode running Ubuntu 18.04.' -authors: ["Gardiner Bryant"] -contributors: ["Gardiner Bryant"] -published: 2019-08-07 -modified: 2020-12-04 -keywords: ["Jellyfin", "Media Server", "PLEX"] -tags: ["ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- -Jellyfin is an open source media library management and streaming platform, similar to [Plex](https://www.plex.tv/). This document guides you through the process of installing and configuring **Jellyfin** on your Linode running Ubuntu 18.04. - -In this guide you complete the following: - -- [Install and configure Jellyfin on a Linode](#install-jellyfin) -- [Create a Reverse Proxy for Jellyfin](#create-a-reverse-proxy-for-jellyfin) - -## Before you Begin - -1. If you have not set up your Linode yet, check out our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode’s hostname and timezone. - -2. Follow up with our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to create a standard user account with `sudo` privileges. - -3. Run the following command to upgrade your packages: - - sudo apt-get update && sudo apt-get upgrade - - {{< note respectIndent=false >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. - {{< /note >}} - -## Install Jellyfin - -1. Install and enable HTTPS transport for APT: - - sudo apt install apt-transport-https - -2. Enable the Universe repository for all of the `ffmpeg` dependencies: - - sudo add-apt-repository universe - -3. Import the GPG signing keys from the Jellyfin team: - - wget -O - https://repo.jellyfin.org/ubuntu/jellyfin_team.gpg.key | sudo apt-key add - - -4. Create a new file located at `/etc/apt/sources.list.d/jellyfin.list` - - sudo touch /etc/apt/sources.list.d/jellyfin.list - -5. Add the Jellyfin `apt` repository to your Linode. - - echo "deb [arch=$( dpkg --print-architecture )] https://repo.jellyfin.org/ubuntu $( lsb_release -c -s ) main" | sudo tee /etc/apt/sources.list.d/jellyfin.list - - {{< note respectIndent=false >}} -Current supported releases are `Cxenial`, `bionic`, `cosmic`, and `disco`. Since we're using Ubuntu 18.04, `lsb_release` becomes `bionic`. -{{< /note >}} - - The output, and the content of the `/etc/apt/sources.list.d/jellyfin.list`, should look something like this: - - {{< output >}} -deb [arch=amd64] https://repo.jellyfin.org/ubuntu bionic main -{{< /output >}} - -6. Finally, update your packages and install Jellyfin - - sudo apt update && sudo apt install jellyfin - -## Configure Jellyfin - -Now that Jellyfin is successfully installed, it needs to be configured and pointed to our media. - -### Initial Setup - -1. Setting up Jellyfin is done through the web interface. Before you can access the web interface, disconnect from SSH and create a secure tunnel via SSH from your local host to your Linode. - - ssh user@192.0.2.1 -L 8888:localhost:8096 - - {{< note respectIndent=false >}} -Substitute `user` with the `sudo user` on your Linode, and `192.0.2.1` with your Linode's IP address. -{{< /note >}} - -1. Open your browser and navigate to `http://localhost:8888/`. You should now see the Jellyfin first-time configuration screen. Start by selecting your preferred language from the drop down menu. Then click the **Next** button to continue. - - ![Language Setup](jellyfin-language-set-up.png) - -1. Create your user account and password. Then click the **Next** button to continue. - - ![Account Creation](jellyfin-account-creation.png) - -1. Now you create the directories to store your media on your Linode. For example, if you want to have music and movies on your server, you would create a directory to store them by using the following command in your terminal: - - cd ~/ - sudo mkdir -p jellyfin-media/music && sudo mkdir jellyfin-media/movies - -1. Back in your browser, now that your account is created, we can add your media. Click on the **Add Media Library** button to begin this process. - - ![Library Setup](jellyfin-library-setup.png) - - {{< note respectIndent=false >}} -Each kind of content type provides a different set of options for you to configure, such as where you would like your metadata retrieved from, etc. -{{< /note >}} - -1. Content in Jellyfin is organized into Libraries. Libraries can have multiple directories from which they aggregate their media. You can specify directories using the Folders plus **(+)** button. Click the **(+)** button to add the folder you created earlier. - - ![Choose the "Folders" button](jellyfin-library-folders.png) - -1. In the **Folders** field, enter the full path to your folder (`/home/username/jellyfin-media/movies`) then click the **Ok** button. - - ![Path to Folder](jellyfin-folder-library.png) - -1. You can add as many libraries as you'd like both now and [later through your dashboard](#add-and-organize-media) - Click the blue **Next** button to proceed to the next sections. - -1. Select your preferred metadata Language, then click on the **Next** button. - -1. Disable port mapping by unchecking the *Enable automatic port mapping* option as this feature can pose a security risk in a cloud environment. Port Mapping is generally enabled in a local environment behind a home router, where you may want your Jellyfin server to be able to seamlessly connect to other devices. - - ![Library Dashboard](jellyfin-disable-port-mapping.png) - -1. Click the **Next** button. Your setup is now complete and you are required to sign in as the user with the password you setup earlier. - -### Disable Unneeded Features (Recommended) - -[DLNA](https://en.wikipedia.org/wiki/Digital_Living_Network_Alliance) is a protocol that incorporates [Universal Plug and Play](https://en.wikipedia.org/wiki/Universal_Plug_and_Play) (or UPnP) standards for digital media sharing across devices. Port `1900` is openly available and any DLNA device or application can have full unrestricted access to your content. Therefore, we recommend disabling DLNA if you are not using it. - -Click the "hamburger" menu in the top left corner of Jellyfin and choose *Dashboard*. Then on the left side of the screen choose *DLNA*, then disable and save your DLNA settings. - -![Disable DLNA](DLNAremoval.png) - -### Add and Organize Media - -- You can add as many libraries as you'd like through the *Dashboard* under *Libraries* at any time. - - ![Library Dashboard](jellyfin-library-dashboard.png) - -- Media can be added to individual folders from inside your Linode using various [file transfer tools](/cloud/guides/tools-reference/file-transfer) and [download methods](/cloud/guides/download-resources-from-the-command-line-with-wget). -- Once files in a folder are added to your Jellyfin server, they can be accessed from your *Home Menu* by clicking on the Home icon at top left of the page after selecting the hamburger menu. - - ![Home Menu](homemenu.png) - -## Create a Reverse Proxy for Jellyfin - -Jellyfin primarily works as a web frontend for your media. That means you generally want to proxy the default Jellyfin websocket to requests. Jellyfin supports a [large number of server software solutions](https://jellyfin.org/docs/general/networking/apache.html) for this purpose, though in this guide, the example is [Apache](http://httpd.apache.org/). - -1. Install Apache with the following command: - - sudo apt install apache2 - -1. Enable proxy settings for Apache with the following commands: - - sudo a2enmod proxy - sudo a2enmod proxy_http - -1. Open a new virtual host file for your configuration. Replace `example.com` in this example with the domain name you are using: - - sudo nano /etc/apache2/sites-available/jellyfin.example.com.conf - - {{< note respectIndent=false >}} -Although nano is used in this example, feel free to use the text editor of your choice. -{{< /note >}} - -1. Use the following Apache virtual host configuration to create your reverse proxy. Replace `jellyfin.example.com` with your domain/subdomain. - - {{< file "/etc/apache2/sites-available/jellyfin.example.com.conf" >}} - - ServerName jellyfin.example.com - ErrorLog /var/log/apache2/jellyfin-error.log - CustomLog /var/log/apache2/jellyfin-access.log combined - - ProxyPreserveHost On - - ProxyPass "/embywebsocket" "ws://127.0.0.1:8096/embywebsocket" - ProxyPassReverse "/embywebsocket" "ws://127.0.0.1:8096/embywebsocket" - - ProxyPass "/" "http://127.0.0.1:8096/" - ProxyPassReverse "/" "http://127.0.0.1:8096/" - -{{< /file >}} - -1. Enable your new website: - - sudo a2ensite jellyfin.example.com.conf - -1. Restart Apache to fully enable your settings: - - sudo systemctl restart apache2 - -You may also want to [set up SSL encryption for this virtual host](/cloud/guides/secure-http-traffic-certbot). For more information regarding this configuration, see Jellyfin's [reverse proxy documentation](https://jellyfin.org/docs/general/networking/index.html#running-jellyfin-behind-a-reverse-proxy) diff --git a/docs/guides/applications/media-servers/how-to-install-shoutcast-dnas-server-on-linux/index.md b/docs/guides/applications/media-servers/how-to-install-shoutcast-dnas-server-on-linux/index.md deleted file mode 100644 index 1ee93ed1ee4..00000000000 --- a/docs/guides/applications/media-servers/how-to-install-shoutcast-dnas-server-on-linux/index.md +++ /dev/null @@ -1,332 +0,0 @@ ---- -slug: how-to-install-shoutcast-dnas-server-on-linux -title: How to Install A SHOUTcast DNAS Server on Linux -description: 'This tutorial will guide you through setup and configuration of a SHOUTcast DNAS server for media streaming on Linux.' -authors: ["Chris Ciufo"] -contributors: ["Chris Ciufo"] -published: 2012-06-07 -modified: 2019-03-25 -keywords: ["shoutcast", " internet radio", " streaming media", " streaming audio"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[SHOUTcast Home Page](http://www.shoutcast.com)' - - '[SHOUTcast Getting Started Guide](http://wiki.winamp.com/wiki/SHOUTcast_Getting_Started_Guide)' - - '[SHOUTcast Broadcast Tools](http://www.shoutcast.com/broadcast-tools)' - - '[SHOUTcast Transcoder MP3 Licensing](http://wiki.winamp.com/wiki/SHOUTcast_DNAS_Transcoder_2#Registering_for_MP3_Stream_Encoding)' -dedicated_cpu_link: true ---- - -![How to Install A SHOUTcast DNAS Server on Linux](install-shoutcast-dnas-title.jpg "How to Install A SHOUTcast DNAS Server on Linux title graphic") - -SHOUTcast is software designed for streaming media over the internet. The SHOUTcast system uses a classic client-server configuration. You can install SHOUTcast on your server and use it to broadcast a stream of music to clients connected to the server. A Shoutcast media server could benefit from large amounts of disk space, so consider using our [Block Storage](https://techdocs.akamai.com/cloud-computing/docs/block-storage) service with this setup. - -{{< note >}} -Be sure to [check the broadcast tools download page](http://www.shoutcast.com/broadcast-tools) for the newest version of SHOUTcast. -{{< /note >}} - -## SHOUTcast DNAS Software - -The SHOUTcast DNAS (Distributed Network Audio Server) software is the server version of the software that allows you to broadcast to listeners. To use SHOUTcast, you'll need to download and install SHOUTcast DNAS on your Linode. - -### Download and Install SHOUTcast - -There are several versions of this software, so make sure you are downloading the correct one. The Linux version is offered in both 32-bit and 64-bit versions. You'll need to [download](http://www.shoutcast.com/broadcast-tools) whichever version corresponds to the operating system you have installed on your Linode. - -1. Create a user for running SHOUTcast, so you are not running it as root. Enter the following command: - - adduser shoutcast - -2. Change to the new user's home directory by entering the following command: - - cd /home/shoutcast - -3. Create a directory for SHOUTcast: - - mkdir sc - -4. Now we can download the DNAS package. In this example, we will download the 32-bit version: - - wget http://download.nullsoft.com/shoutcast/tools/sc_serv2_linux_x64-latest.tar.gz - -5. Extract the SHOUTcast files to the new directory by entering the following command: - - tar -xzf sc_serv2_linux_x64-latest.tar.gz -C sc - -6. Change the ownership from `root` to the SHOUTcast user: - - chown -R shoutcast.shoutcast /home/shoutcast/sc - -The SHOUTcast DNAS software is now installed on your Linode. - -### Configure SHOUTcast - -Now, you'll want to modify the configuration. This is necessary to specify passwords and set the SHOUTcast port. Here's how to configure SHOUTcast: - -1. Open the SHOUTcast configuration file: - - nano sc/sc_serv_basic.conf - -2. This will bring up the configuration file for editing, as shown below. - - {{< file "/home/shoutcast/sc/sc_serv_basic.conf" conf >}} -; NOTE: for any relative paths specified are relative to -; sc_serv and not to where the conf file is being stored - -; here we will setup where the log and other related files -; will be stored. make sure that these folders exist else -; sc_serv will throw an error and will close itself down. -; we will make the logs save to the sc_serv2 directory -logfile=logs/sc_serv.log -w3clog=logs/sc_w3c.log -banfile=control/sc_serv.ban -ripfile=control/sc_serv.rip - - -; for testing we will make the server only work locally -; (i.e. localhost / 127.0.0.1) though if this is left out -; or set to publicserver=always then we attempt to make a -; connection to the YP for listing - do not forget to add -; in a 'streamauthhash' value for any public streams made -;publicserver=never - - -; if you're wanting to use a different port to use for any -; connections then you can use this option e.g. to use 80 -; otherwise port 8000 is used as the default to listen on. -;portbase=80 - - -; password used by sc_trans or the Winamp dsp plug-in -; NOTE: remember to change this to something else -password=testing - - -; password used for accessing the administration pages -; NOTE: remember to change this to something else -adminpassword=changeme - - -; now we will specify the details of the stream we're going -; to serve which can be done as follows -streamid=1 -streampath=/test.aac - -; or - -; it can be done like this which is how it needs to be done -; if you are going to provide multiple streams from sc_serv -;streamid_1=1 -;streampath_1=/test.aac -;streamid_2=2 -;streampath_2=/test2.aac - -{{< /file >}} - - -3. Set the `password` and `adminpassword` variables to whatever you want them to be. -4. The `portbase` variable should be set to use a port you are not using for anything else. The default port for SHOUTcast is 8000. - - {{< note respectIndent=false >}} -If you set the `portbase` variable to anything besides 8000, be sure to uncomment it by deleting the semicolon in front of the variable. -{{< /note >}} - -5. Save the changes to the SHOUTcast configuration file by pressing Control-X, and then Y. - -Now that the configuration is set and saved, we can start the server. - -### Start SHOUTcast - -Now, you can start the SHOUTcast server. Here's how: - -1. You'll want to run your shoutcast in a [screen session](/cloud/guides/using-gnu-screen-to-manage-persistent-terminal-sessions). Let's jump into a screen session by entering the following command: - - screen - -2. Start the SHOUTcast server by entering the following command: - - ./sc_serv sc_serv_simple.conf - -3. After you issue the start command, you should see the startup output ending with: - - 2011-11-02 14:50:03 I msg:[MICROSERVER] Listening for connection on port 8000 - 2011-11-02 14:50:03 I msg:[MICROSERVER] Listening for connection on port 8001 - -4. You can detach from your screen session at this point. To do so, hold down the Control key and press A, let go, and then press D. -5. You should be back at the command prompt outside of your screen session. If you need to reattach later on, simply type: - - screen -raAd - -Your SHOUTcast server is now running! You can now connect to it and begin your broadcast. - -## SHOUTcast Transcoder - -The SHOUTcast Transcoder allows you to schedule DJ play times, broadcast an automatic playlist in a specific time slot, schedule time slots for relayed broadcasts, etc. - -{{< note >}} -To encode your streams in MP3 format, you *must* [purchase a license key from WinAmp, which costs \$5 USD](http://wiki.winamp.com/wiki/SHOUTcast_DNAS_Transcoder_2#Registering_for_MP3_Stream_Encoding). -{{< /note >}} - -### Download and Install SHOUTcast Transcoder - -We'll use the same shoutcast user to set up the Transcoder software. Here's how to download and install the transcoder: - -1. Change directories by entering the following command: - - cd /home/shoutcast - -2. Create a new directory for the transcoder by entering the following command: - - mkdir sct - -3. Download the SHOUTcast transcoder archive by entering the following command: - - wget http://download.nullsoft.com/shoutcast/tools/sc_trans_linux_10_07_2011.tar.gz - -4. Extract the SHOUTcast transcoder files by entering the following command: - - tar -xzf sc_trans_linux_10_07_2011.tar.gz -C sct - -5. Change ownership from `root` to the SHOUTcast user: - - chown -R shoutcast.shoutcast /home/shoutcast/sct - -6. Change directories by entering the following command: - - cd sct - -7. Change permissions by entering the following command: - - chmod a+x sc_trans - -The SHOUTcast transcoder is now installed on your Linode. - -### Configure the SHOUTcast Transcoder - -This example will walk you through a basic configuration. - -1. Open the configuration file by entering the following command: - - nano /home/shoutcast/sct/sc_trans_basic.conf - -2. You can modify the bitrate to change the sound quality of the music and limit the amount of bandwidth consumed. If you purchase MP3 licensing, you can modify the encoder section to add the MP3 encoding and your unlock data: - - {{< file "/home/shoutcast/sct/sc_trans_basic.conf" conf >}} -; for testing we will only setup a single encoder though it -; is easy to add in additional encoder configurations and -; we are using an aac plus encoder as the default due to -; the licensing requirements for mp3 encoding as detailed -; in sc_trans.txt - section 2.5). -encoder_1=aacp -encoder_2=mp3 -bitrate_1=56000 -bitrate_2=56000 - -unlockkeyname=YourUnlockName -unlockkeycode=YourUnlockCode - -{{< /file >}} - - -3. Next, modify the `sc_trans` to `sc_serv` connection details: - - {{< file "/home/shoutcast/sct/sc_trans_basic.conf" conf >}} -; this is where we define the details required for sc_trans -; to connect to the sc_serv instance being used where the -; details must match those specified in sc_serv_basic.conf -outprotocol_1=3 -serverip_1=127.0.0.1 -; default is 8000, if not change to sc_serv's 'portbase' -serverport_1=8000 -; this is the same as 'password' in sc_serv_basic.conf -password_1=testing -; this is the same as 'streamid' in sc_serv_basic.conf for -; the stream we are acting as the source for -streamid_1=1 -; this is a name for the source we're creating and is used -; with the AJAX control api or can be left blank to get a -; generic name created in the form of 'endpointX' where 'X' -; is the index of the created source from sc_trans lists. -endpointname_1=/Bob - -{{< /file >}} - - -4. This step is optional, but you can also update your stream information: - - {{< file "/home/shoutcast/sct/sc_trans_basic.conf" conf >}} -; here you would provide any information to fill in details -; provided to clients about the stream. it us up to you what -; is entered though do not do anything which will annoy, etc -streamtitle=My Test Server -streamurl=http://www.shoutcast.com -genre=Misc - -{{< /file >}} - - -5. Set your playlist file for an automated stream: - - {{< file "/home/shoutcast/sct/sc_trans_basic.conf" conf >}} -; here we specify a playlist to use as the master list from -; which to play files from. -playlistfile=playlists/main.lst -{{< /file >}} - - -6. Now set the port, username, and password for the transcoder admin panel access: - - {{< file "/home/shoutcast/sct/sc_trans_basic.conf" conf >}} -; these options will allow you access the admin interfaces -; of sc_trans though also allows the 'testui' example to be -; accessed. remember to change the password, etc as needed -adminport=7999 -adminuser=admin -adminpassword=goaway - -{{< /file >}} - - -7. Save the changes to the SHOUTcast configuration file by pressing Control-X, and then Y. -8. If you are using an automated playlist, upload your music files to the `/home/shoutcast/sct/music` directory. -9. If you are using an automated playlist, you'll also need to create a playlist file. Here is an example: - - {{< file "/home/shoutcast/sct/playlists/playlist.lst" >}} -# This example playlist is used as the main playlist sc_trans will use to pick -# out the files it will use to create its output for the source we generate. -# -# Also remember to use the correct path format for the OS you are using and to -# ensure that the files you want to play are present in the location you choose -# e.g. -../music/shoutcast.mp3 -# -# In this example we will just assume that all of the files associated to the -# playlist are in one folder and all have an mp3 extension though there is no -# reason why you cannot explicitly specify files to use or to reference a tool. -# See sc_trans.txt - section 7.1 for more information on how playlists work. - -# Remember to change this to reference the files you want to use when trying -# the sc_trans_playlist.conf example which is best tried with full length files - -{{< /file >}} - - -### Start SHOUTcast Transcoder - -Once you have the transcoder configured and ready to go, you need to start it. To run the transcoder as a daemon, simply run this command, substituting `sc_trans_basic.conf` for whatever configuration file you are using: - - ./sc_trans daemon ./sc_trans_basic.conf - -If no errors appear, you should see output similar to the line below, where XXXX is the PID: - - sc_trans going daemon with PID [XXXX] - -To shut down the transcoder, you'll just need to issue a kill command: - - kill -15 PID - -## SHOUTcast Source DSP - -SHOUTcast's Source DSP plugin was developed for use with WinAmp version 5.5 and newer. This plugin gives you the ability to use WinAmp as a source for your sc\_serv (DNAS) or sc\_trans (Transcoder). It will also allow you to capture an audio input from your sound card and its line-in or microphone inputs. You will need a working installation of either the DNAS by itself, or the Transcoder feeding into a DNAS installation, before you can use the DSP WinAmp plugin. The download for the DSP plugin is near the bottom of the [broadcast tools page](http://www.shoutcast.com/broadcast-tools). - -Instructions for installation and configuration are located in the [WinAmp wiki](http://wiki.winamp.com/wiki/Source_DSP_Plug-in#Installing_the_Plug-in). diff --git a/docs/guides/applications/media-servers/install-plex-media-server-on-centos-7/index.md b/docs/guides/applications/media-servers/install-plex-media-server-on-centos-7/index.md deleted file mode 100644 index a6241153b9f..00000000000 --- a/docs/guides/applications/media-servers/install-plex-media-server-on-centos-7/index.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -slug: install-plex-media-server-on-centos-7 -title: Install Plex Media Server on CentOS 7 -description: 'This guide shows you how to install Plex Media Server, an application that organizes and can stream your photos, videos, music, and more, on CentOS 7.' -authors: ["Nick Brewer"] -contributors: ["Nick Brewer"] -published: 2017-05-03 -modified: 2019-01-31 -keywords: ["plex", "media", "centos"] -tags: ["centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Plex Media Server Documentation](https://support.plex.tv/hc/en-us/categories/200007567-Plex-Media-Server)' -dedicated_cpu_link: true -relations: - platform: - key: how-to-install-plex - keywords: - - distribution: CentOS 7 -aliases: [] ---- - -[Plex](https://www.plex.tv/) is a feature-rich media library platform that allows you to organize and stream your digital video and audio from virtually anywhere. Basic Plex features are [free](https://support.plex.tv/articles/202526943-plex-free-vs-paid/), while the paid Plex Pass adds additional features. - -This guide demonstrates how to set up **Plex Media Server** on a Linode running CentOS 7, and how to connect client devices. A Plex media server could benefit from large amounts of disk space, so consider using Linode's [Block Storage](https://techdocs.akamai.com/cloud-computing/docs/block-storage) service with this setup. - -![Install Plex Media Server on CentOS 7](install-plex-media-server-on-centos-7.png) - - -## Before you Begin - -- You will need root access to your Linode, or a [limited user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account) with `sudo` privilege. - -- Set your system's [hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and [time zone](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#set-the-timezone). - -- Plex requires that you create an [account](https://www.plex.tv/features/) to make use of the service, and provides additional features such as DVR capability and offline viewing if you pay for their premium [Plex Pass](https://www.plex.tv/features/plex-pass/) offering. To complete this guide, you will need a Plex account. - - -## Install and Configure Plex - -1. Go to [Plex's download page](https://www.plex.tv/media-server-downloads/). Select *Linux*, and then click *Choose Distribution*. - - ![Plex download button](plex-download-button.png) - -2. In the menu that appears, right-click on *CentOS 64-bit (RPM for CentOS 7 or newer)* and copy the download link. Use `cURL` to download the .rpm package directly to your Linode. The example below uses the current link for CentOS at the time of writing. Be sure you install the most recent version of Plex. - - curl -O https://downloads.plex.tv/plex-media-server/1.14.1.5488-cc260c476/plexmediaserver-1.14.1.5488-cc260c476.x86_64.rpm - -3. Update your system and install Plex: - - sudo yum update - sudo yum install plexmediaserver*.rpm - -4. Enable Plex Media Server to start on reboot and then start the server: - - sudo systemctl enable plexmediaserver - sudo systemctl start plexmediaserver - -5. Finally, you'll create the directories that will store your Plex media. In this example we'll create library directories for `movies` and `television` within a `plex-media` directory. These will be located within your user's `/home`: - - cd ~/ - mkdir -p plex-media/movies && mkdir plex-media/television - -6. Administration of the Plex server is performed via its web interface. Before you can connect to the web interface from your workstation, you will first need to create an SSH tunnel to your Linode. Substitute `user` with the `sudo` user on your Linode, and `192.0.2.0` with its IP address. - - ssh user@192.0.2.0 -L 8888:localhost:32400 - -7. Go to `http://localhost:8888/web` in a web browser and log in to Plex. - - ![Plex web interface.](plex-browser-view.png) - -8. Give your Plex server a name. Be sure to leave the **Allow me to access my media outside my home** box checked, and click **Next**: - - ![Plex web interface - Server Name.](plex-server-name.png) - - -## Add and Organize Media - -1. Now that you've signed into Plex, you should see the following page. Click the **Add Library** button to start setting up your media libraries. - - ![Plex web interface - Add Library](plex-add-library.png) - -1. Select your library type, and click **Next**. - - ![Plex web interface - Library type](plex-library-type.png) - -1. Navigate to the corresponding media directory that you created previously, then click **Add**. - - ![Plex web interface - Library location](plex-library-location.png) - -1. You can add additional libraries by clicking the **+** symbol next to the **Libraries** list on the Plex side bar: - - ![Plex web interface - additional Library](plex-additional-library.png) - -1. Add your media to the appropriate directories. Be sure to review Plex's [naming conventions](https://support.plex.tv/hc/en-us/categories/200028098-Media-Preparation) for media files to ensure that your files are identified correctly. - -## Disable DLNA (Recommended) - -[DLNA](https://en.wikipedia.org/wiki/Digital_Living_Network_Alliance) is a protocol that incorporates [Universal Plug and Play](https://en.wikipedia.org/wiki/Universal_Plug_and_Play) (or UPnP) standards for digital media sharing across devices. Any DLNA device or application can have full unrestricted access to your Plex content, so disable DLNA if you won't be using it. - -From the Plex web interface, click the wrench icon in the upper right corner and then click **DLNA** in the left sidebar. Uncheck **Enable the DLNA server** and click **Save Changes**. - -![Plex media client](plex-dlna-disable.png) - -## Connect to your Plex Server - -Now that your server is set up, you're ready to connect to it from a Plex client. Plex is supported by a number of different platforms and you can find a full list of client applications [here](https://support.plex.tv/hc/en-us/categories/200006953-Plex-Apps). - -The examples provided here will use Plex Media Player for macOS. - -1. [Download](https://www.plex.tv/downloads/) the appropriate media player application or install it via your device's app store. - -1. Sign in to the Plex client app using the same Plex account as your server. - -1. Your Plex client will have a dropdown menu where you can select your server. Once it's selected, you can navigate to the library with the content that you wish to view: - - ![Plex media client](plex-media-client.png) diff --git a/docs/guides/applications/media-servers/install-plex-media-server-with-salt/index.md b/docs/guides/applications/media-servers/install-plex-media-server-with-salt/index.md deleted file mode 100644 index ca8d1cc469c..00000000000 --- a/docs/guides/applications/media-servers/install-plex-media-server-with-salt/index.md +++ /dev/null @@ -1,249 +0,0 @@ ---- -slug: install-plex-media-server-with-salt -title: "Installing Plex Media Server on Ubuntu 18.04 Using Salt Masterless" -title_meta: "How to Install Plex Media Server on Ubuntu 18.04 Using Salt" -description: 'This guide shows you how to install Plex Media Server, an application that organizes and can stream your photos, videos, music, and more, using Salt.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2019-01-31 -modified: 2019-01-02 -keywords: ['plex','media','server','ubuntu 18.04','ubuntu','salt','saltstack'] -tags: ["ubuntu", "salt"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Salt Masterless Walkthough](https://docs.saltstack.com/en/latest/topics/tutorials/quickstart.html)' - - '[Salt Fileserver Backend Walthrough](https://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html)' - - '[Plex Media Server Quick State](https://support.plex.tv/articles/200264746-quick-start-step-by-step-guides/)' -dedicated_cpu_link: true -aliases: [] ---- - -Plex is a media server that allows you to stream video and audio content that you own to many different types of devices. In this guide you will learn how to use a masterless Salt minion to set up a Plex server, attach and use a Block Storage Volume, and how to connect to your media server to stream content to your devices. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's hostname and timezone. - -1. Follow the steps in the [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide. - -1. Update your system: - - sudo apt-get update && sudo apt-get upgrade - -2. You will need to create a Block Storage Volume and attach it to your Linode. You will format and mount the drive as part of this guide. This volume will be used to store your media, so you should pick a size that's appropriate for your media collection, though you can resize the volume later if you need more storage. For more on Block Storage, see our [Block Storage Overview](https://techdocs.akamai.com/cloud-computing/docs/block-storage) guide. - -3. Plex requires an account to use their service. Visit the [Plex website](https://www.plex.tv/) to sign up for an account if you do not already have one. - -{{< note >}} -The steps in this guide require root privileges. Be sure to run the steps below with the `sudo` prefix. For more information on privileges, see our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Prepare the Salt Minion - -1. On your Linode, create the `/srv/salt` and `/srv/pillar` directories. These are where the Salt state files and Pillar files will be housed. - - mkdir /srv/salt && mkdir /srv/pillar - -1. Install `salt-minion` via the Salt bootstrap script: - - curl -L https://bootstrap.saltstack.com -o bootstrap_salt.sh - sudo sh bootstrap_salt.sh - -1. The Salt minion will use the official [Plex Salt Formula](https://github.com/saltstack-formulas/plex-formula), which is hosted on the SaltStack GitHub repository. In order to use a Salt formula hosted on an external repository, you will need GitPython installed. Install GitPython: - - sudo apt-get install python-git - -## Modify the Salt Minion Configuration - -1. Because the Salt minion is running in masterless mode, you will need to modify the minion configuration file (`/etc/salt/minion`) to instruct Salt to look for state files locally. Open the minion configuration file in a text editor, uncomment the line `#file_client: remote`, and set it to `local`: - - {{< file "/etc/salt/minion" yaml >}} -... - -# Set the file client. The client defaults to looking on the master server for -# files, but can be directed to look at the local file directory setting -# defined below by setting it to "local". Setting a local file_client runs the -# minion in masterless mode. -file_client: local - -... -{{< /file >}} - -1. There are some configuration values that do not normally exist in `/etc/salt/minion` which you will need to add in order to run your minion in masterless mode. Copy the following lines into the end of `/etc/salt/minion`: - - {{< file "/etc/salt/minion" yaml >}} -... - -fileserver_backend: - - roots - - gitfs - -gitfs_remotes: - - https://github.com/saltstack-formulas/plex-formula.git - -gitfs_provider: gitpython -{{< /file >}} - - The `fileserver_backend` block instructs the Salt minion to look for Salt configuration files in two places. First, it tells Salt to look for Salt state files in our minion's `roots` backend (`/srv/salt`). Secondly, it instructs Salt to use the Git Fileserver (gitfs) to look for Salt configuration files in any Git remote repositories that have been named in the `gitfs_remotes` section. The address for the Plex Salt formula's Git repository is included in the `gitfs_remotes` section. - - {{< note respectIndent=false >}} -It is best practice to create a fork of the Plex formula's Git repository on GitHub and to add your fork's Git repository address in the `gitfs_remotes` section. This will ensure that any further changes to the upstream Plex formula which might break your current configuration can be reviewed and handled accordingly, before applying them. -{{< /note >}} - - Lastly, GitPython is specified as the `gitfs_provider`. - -## Create the Salt State Tree - -1. Create a Salt state top file at `/srv/salt/top.sls` and copy in the following configuration. This file tells Salt to look for state files in the plex folder of the Plex formula's Git repository, and for a state files named `disk.sls` and `directory.sls`, which you will create in the next steps. - - {{< file "/srv/salt/top.sls" yaml >}} -base: - '*': - - plex - - disk - - directory -{{< /file >}} - - -1. Create the `disk.sls` file in `/srv/salt`: - - {{< file "/srv/salt/disk.sls" yaml >}} -disk.format: - module.run: - - device: /dev/disk/by-id/scsi-0Linode_Volume_{{ pillar['volume_name'] }} - - fs_type: ext4 - -/mnt/plex: - mount.mounted: - - device: /dev/disk/by-id/scsi-0Linode_Volume_{{ pillar['volume_name'] }} - - fstype: ext4 - - mkmnt: True - - persist: True -{{< /file >}} - - This file instructs Salt to prepare your Block Storage Volume for use with Plex. It first formats your Block Storage Volume with the `ext4` filesystem type by using the `disk.format` Salt module, which can be run in a state file using `module.run`. Then `disk.sls` instructs Salt to mount your volume at `/mnt/plex`, creating the mount target if it does not already exist with `mkmnt`, and persisting the mount to `/etc/fstab` so that the volume is always mounted at boot. - -1. Create the `directory.sls` file in `/srv/salt`: - - {{< file "/srv/salt/directory.sls" >}} -/mnt/plex/plex-media: - file.directory: - - require: - - mount: /mnt/plex - - user: username - - group: plex - -/mnt/plex/plex-media/movies: - file.directory: - - require: - - mount: /mnt/plex - - user: username - - group: plex - -/mnt/plex/plex-media/television: - file.directory: - - require: - - mount: /mnt/plex - - user: username - - group: plex -{{< /file >}} - - The directories that are created during this step are for organizational purposes, and will house your media. Make sure you replace `username` with the name of the limited user account you created when following the [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide. The location of the directories is the volume you mounted in the previous step. If you wish to add more directories, perhaps one for your music media, you can do so here, just be sure to include the `- require` block, as this prevents Salt from trying to create the directory before the Block Storage Volume has been mounted. - -1. Go to the [Plex Media Server download page](https://www.plex.tv/media-server-downloads/#plex-media-server) and note the most recent version of their Linux distribution. At the time of writing, the most recent version is `1.13.9.5456-ecd600442`. Create the `plex.sls` Pillar file in `/srv/pillar` and change the Plex version number and the name of your Block Storage Volume as necessary: - - {{< file "/srv/pillar/plex.sls" yaml >}} -plex: - version: 1.13.9.5456-ecd600442 -volume_name: plex -{{< /file >}} - -1. Create the Salt Pillar top file in `/srv/pillar`. This file will instruct Salt to look for the `plex.sls` Pillar file you created in the previous step. - - {{< file "/srv/pillar/top.sls" >}} -base: - '*': - - plex -{{< /file >}} - -1. Apply your Salt state locally using `salt-call`: - - salt-call --local state.apply - - You should see a list of the changes Salt applied to your system. You have successfully installed Plex using Salt. - -## Set Up Plex - -### Initial Configuration - -1. You'll need to create an SSH tunnel to your Linode to connect to Plex's web UI. On your local computer, run the following command, replacing `` with your Plex server's IP address.: - - ssh username@ -L 8888:localhost:32400 - -1. In a browser, navigate to `http://localhost:8888/web/`. - -1. Sign in with your Plex username and password. - -1. Name your media server. This example uses the name `linode-plex`. Be sure to check the box that reads *Allow me to access my media outside my home* and then click **Next**. - - ![Name your media server](plex-salt-name-server.png) - -### Organize Your Media - -1. Click on the **Add Library** button: - - ![Click on Add Media](plex-salt-organize-media.png) - -2. Select *Movies* and click **Next**: - - ![Select Movies and click next](plex-salt-add-library1.png) - -3. Click **Browse for Media Folder** and select the appropriate folder at `/mnt/plex/plex-media/movies`. Then click **Add**: - - ![Select the appropriate folder](plex-salt-add-library2.png) - -4. Repeat the process to add your 'Television' folder. - -5. When you are done adding your libraries, click **Add Library**. - -5. To continue the configuration process, click **Next**. - -6. Click on **Get Plex Apps** to download the appropriate Plex client for your device. Then click **Done**. - - ![Download the appropriate client for your device](plex-salt-download-app.png) - -7. In the future you can add more libraries by hovering over the menu and clicking the plus sign (+) next to *LIBRARIES*. - - ![Add more libraries](plex-salt-add-library3.png) - -### Disable DLNA (Recommended) - -[DLNA](https://en.wikipedia.org/wiki/Digital_Living_Network_Alliance) is a protocol that incorporates [Universal Plug and Play](https://en.wikipedia.org/wiki/Universal_Plug_and_Play) (or UPnP) standards for digital media sharing across devices. If you do not wish to make use of it, it’s recommended that you disable this feature, as it is openly connectable on port `1900`. From the Plex web interface, click the wrench icon in the upper right corner, and navigate to the **DLNA** section under *SETTINGS*. Uncheck *Enable the DLNA server*, and click **Save Changes**: - -![Disable DLNA](plex-salt-disable-dlna.png) - -## Connect to Your Plex Server - -1. Visit the [Plex Apps](https://www.plex.tv/media-server-downloads/#plex-app) download page or the app store on your device to download Plex Media Player if you have not already done so. - -1. Open your Plex app. The example provided here will use the Plex Media Player for macOS. - -1. Sign in to Plex. - -1. On the left there's a dropdown menu where you can select your server by the name you chose. Select your server. - - ![Connect to your Plex Server](plex-salt-select-remote.png) - -1. You are now able to stream your content with Plex. - - ![Plex's macOS App](plex-salt-mac-app.png) - -## Transfer Media to Your Server - -1. You can use SCP to transfer media to your server from your local computer. Replace your username and `123.456.7.8` with the IP address of your Linode. - - scp example_video.mp4 username@123.456.7.8:/mnt/plex/plex-media/movies - -1. Once you've transferred files to your Plex media server, you may need to scan for new files before they show up in your Library. Click on the ellipsis next to a Library and select **Scan Library Files**. - - ![Scan your Library for new files](plex-salt-scan-for-files.png) diff --git a/docs/guides/applications/media-servers/install-subsonic-media-server-on-ubuntu-or-debian/index.md b/docs/guides/applications/media-servers/install-subsonic-media-server-on-ubuntu-or-debian/index.md deleted file mode 100644 index 00673a4dfdb..00000000000 --- a/docs/guides/applications/media-servers/install-subsonic-media-server-on-ubuntu-or-debian/index.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -slug: install-subsonic-media-server-on-ubuntu-or-debian -title: "Installing Subsonic Media Server on Ubuntu or Debian to Stream Music" -title_meta: "How to Install Subsonic Media Server on Ubuntu or Debian" -description: 'Subsonic is a free music streaming application. This guide shows how to install Subsonic media server on a Linode.' -authors: ["Alex Fornuto"] -contributors: ["Alex Fornuto"] -published: 2015-02-02 -modified: 2019-01-31 -keywords: ["subsonic", "music", "audio", "streaming", "media server"] -aliases: [] -tags: ["debian", "ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Subsonic official site](http://www.subsonic.org)' -dedicated_cpu_link: true ---- - -![Install Subsonic Media Server on Ubuntu or Debian to Stream Music Through Your Linode](Install_Subsonic_Media_Server_on_Ubuntu_or_Debian_smg.png "Install Subsonic Media Server on Ubuntu or Debian to Stream Music Through Your Linode") - -## What is Subsonic? - -[Subsonic](http://subsonic.org) is an easy-to-use media streaming service with a user-friendly interface and the ability to share music and video with multiple users. It is highly customizable and includes features such as Chromecast support and file conversion. - -This guide shows how to set up Subsonic on a Linode running Debian or Ubuntu. If you have a large music library, consider attaching a [Block Storage Volume](https://techdocs.akamai.com/cloud-computing/docs/block-storage) to your Linode to store your music files. - -## Install Java - -You will need Java installed on your system to run Subsonic. - -{{% content "install-java-8-ppa" %}} - -## Install Subsonic - -1. The latest version of Subsonic (as of publication) is 6.1.3, and the most recent release can always be found on their [download](http://www.subsonic.org/pages/download.jsp) page. Download and install Subsonic onto your Linode: - - wget https://s3-eu-west-1.amazonaws.com/subsonic-public/download/subsonic-6.1.3.deb - sudo dpkg -i subsonic-6.1.3.deb - -2. Subsonic runs as the root user by default, which is insecure. Create a new system user for subsonic to run as: - - sudo useradd --system subsonic - sudo gpasswd --add subsonic audio - -3. Open `/etc/default/subsonic` in a text editor. In this file you can change the user, the port Subsonic listens on, increase the amount of memory it can use, and encrypt your streaming traffic with SSL. Change the `SUBSONIC_USER` variable to the new `subsonic` user: - - {{< file "/etc/default/subsonic" >}} -# Type "subsonic --help" on the command line to read an -# explanation of the different options. -# -# For example, to specify that Subsonic should use port 80 (for http) -# and 443 (for https), and use a Java memory heap size of 200 MB, use -# the following: -# -# SUBSONIC_ARGS="--port=80 --https-port=443 --max-memory=200" - -SUBSONIC_ARGS="--max-memory=150" - -SUBSONIC_USER=subsonic -{{< /file >}} - - {{< note respectIndent=false >}} -If you have a firewall set up, allow connections from the port Subsonic is listening on. -{{< /note >}} - -3. Restart Subsonic: - - sudo systemctl restart subsonic - -## Configuration and Use - -1. By default, Subsonic looks for music files stored in `/var/music`. Create this directory and change its ownership to the `subsonic` user. You can substitute another directory if you would prefer to store your music elsewhere. - - sudo mkdir /var/music - sudo chown subsonic:subsonic /var/music - -2. Open Subsonic in your browser by navigating to port 4040 on your Linode's public IP address or domain name. - -3. The first time you access your Subsonic server in your browser, you will see the following: - - ![First time Subsonic access](subsonic-firstlogin.png) - -4. Use the default username and password (admin/admin) or the provided link to log in: - - ![First time Subsonic access](subsonic-gettingstarted.png) - -5. Create a password for your admin account. You can also set up any other accounts at this time. - - {{< note respectIndent=false >}} -Passwords in the Subsonic database are stored in hex format, but not encrypted. -{{< /note >}} - -6. Click on the **Media folders** link and point Subsonic to where you will store your music. If you used the default directory (`/var/music`) above, you can skip this step. Once you've pointed Subsonic to the correct directory and uploaded your music, you can press **Scan media folders now**. Subsonic will then create a database of music files. - - ![First time Subsonic access](subsonic-foldersetup.png) - -  - -![The Subsonic Interface.](subsonic-setup.png) - -## Next Steps - -Subsonic can be [configured to use SSL](http://www.subsonic.org/pages/getting-started.jsp), or you can use an [NGINX reverse proxy](/cloud/guides/use-nginx-reverse-proxy). diff --git a/docs/guides/applications/messaging/install-znc-debian/index.md b/docs/guides/applications/messaging/install-znc-debian/index.md deleted file mode 100644 index cbd9ccf0e36..00000000000 --- a/docs/guides/applications/messaging/install-znc-debian/index.md +++ /dev/null @@ -1,210 +0,0 @@ ---- -slug: install-znc-debian -title: 'Install ZNC from Source on Debian' -description: 'This guide shows how to install the open-source application ZNC, a IRC bouncer designed to run on a server that remains connected to IRC, on a Linode.' -authors: ["Alex Fornuto"] -contributors: ["Alex Fornuto"] -published: 2014-08-21 -modified: 2015-06-04 -keywords: ["install znc", "irc bouncer", "znc on debian", "configure znc", "znc"] -tags: ["debian"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- - -ZNC is an IRC bouncer. It's designed to run on a server that remains connected to an IRC network and buffer messages. With ZNC, a local IRC client can connect and disconnect without losing a chat session or missing any messages. In this guide, ZNC will be installed from source and then configured. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Before You Begin - -1. Make sure the system is up to date: - - sudo apt-get update && sudo apt-get upgrade -y - -2. Install the `build-essential` and `checkinstall` packages: - - sudo apt-get install build-essential checkinstall - -3. If you want to use SSL encryption to connect to the web interface (recommended), install `libssl-dev`: - - sudo apt-get install libssl-dev - -## Install ZNC - -1. Download the latest version of ZNC (1.6.0 at the time of writing): - - wget http://znc.in/releases/znc-1.6.0.tar.gz - -2. Expand the archive file: - - tar -xvf znc-1.*.tar.gz - -3. Move into the ZNC directory: - - cd znc-1.6.0 - -4. Run the `configure` script to make sure the Linode has all the needed prerequisites: - - ./configure - - If not, you will need to install any missing packages prior to continuing. - -5. Install ZNC: - - make - sudo checkinstall --fstrans=0 make install - - {{< note respectIndent=false >}} -The program `checkinstall` creates a `.deb` package which you can use to reinstall this version of ZNC in the future. It has its own set of options to review. If you prefer, you can instead run `sudo make install` to install ZNC as is. -{{< /note >}} - -## Configure ZNC - - -1. Begin the configuration process: - - znc --makeconf - - -2. This will launch an interactive script asking you for input on a variety of parameters. Below is an example output of the `makeconf` script with standard options selected. To match your needs, you can use or change the provided input at your discretion. If you're not sure, use the default option. Many of these options can be adjusted later through the web interface. - - {{< note respectIndent=false >}} -Make sure to change the `username` variable. -{{< /note >}} - - [ .. ] Checking for list of available modules... - [ >> ] ok - [ ** ] Building new config - [ ** ] - [ ** ] First let's start with some global settings... - [ ** ] - [ ?? ] What port would you like ZNC to listen on? (1025 to 65535): 5678 - [ ?? ] Would you like ZNC to listen using SSL? (yes/no) [no]: yes - [ ?? ] Would you like ZNC to listen using both IPv4 and IPv6? (yes/no) [yes]: - [ .. ] Verifying the listener... - [ >> ] ok - [ ** ] - [ ** ] -- Global Modules -- - [ ** ] - [ ** ] +-----------+----------------------------------------------------------+ - [ ** ] | Name | Description | - [ ** ] +-----------+----------------------------------------------------------+ - [ ** ] | partyline | Internal channels and queries for users connected to znc | - [ ** ] | webadmin | Web based administration module | - [ ** ] +-----------+----------------------------------------------------------+ - [ ** ] And 9 other (uncommon) modules. You can enable those later. - [ ** ] - [ ?? ] Load global module ? (yes/no) [no]: - [ ?? ] Load global module ? (yes/no) [no]: yes - [ ** ] - [ ** ] Now we need to set up a user... - [ ** ] - [ ?? ] Username (AlphaNumeric): user - [ ?? ] Enter Password: - [ ?? ] Confirm Password: - [ ?? ] Would you like this user to be an admin? (yes/no) [yes]: - [ ?? ] Nick [user]: user - [ ?? ] Alt Nick [user_]: - [ ?? ] Ident [user]: - [ ?? ] Real Name [Got ZNC?]: - [ ?? ] Bind Host (optional): - [ ** ] Enabled user modules [chansaver, controlpanel] - [ ** ] - [ ?? ] Set up a network? (yes/no) [yes]: - [ ** ] - [ ** ] -- Network settings -- - [ ** ] - [ ?? ] Name [freenode]: - [ ?? ] Server host [chat.freenode.net]: - [ ?? ] Server uses SSL? (yes/no) [yes]: - [ ?? ] Server port (1 to 65535) [6697]: - [ ?? ] Server password (probably empty): - [ ?? ] Initial channels: - [ ** ] Enabled network modules [simple_away] - [ ** ] - [ .. ] Writing config [/home/elle/.znc/configs/znc.conf]... - [ >> ] ok - [ ** ] - [ ** ] To connect to this ZNC you need to connect to it as your IRC server - [ ** ] using the port that you supplied. You have to supply your login info - [ ** ] as the IRC server password like this: user/network:pass. - [ ** ] - [ ** ] Try something like this in your IRC client... - [ ** ] /server +5678 user: - [ ** ] - [ ** ] To manage settings, users and networks, point your web browser to - [ ** ] https://:5678/ - [ ** ] - [ ?? ] Launch ZNC now? (yes/no) [yes]: - [ .. ] Opening config [/home/elle/.znc/configs/znc.conf]... - [ >> ] ok - [ .. ] Loading global module [webadmin]... - [ >> ] [/usr/local/lib/znc/webadmin.so] - [ .. ] Binding to port [+5678]... - [ >> ] ok - [ ** ] Loading user [user] - [ ** ] Loading network [freenode] - [ .. ] Loading network module [simple_away]... - [ >> ] [/usr/local/lib/znc/simple_away.so] - [ .. ] Adding server [chat.freenode.net +6697 ]... - [ >> ] ok - [ .. ] Loading user module [chansaver]... - [ >> ] ok - [ .. ] Loading user module [controlpanel]... - [ >> ] ok - [ .. ] Forking into the background... - [ >> ] [pid: 27369] - [ ** ] ZNC - 1.6.0 - http://znc.in - - Once you've completed the configuration and launched ZNC, you can access the web interface by going to your Linode's IP address in your web browser. Be sure to specify the port you defined during the configuration script and prefix it with `https://` . - - {{< note respectIndent=false >}} -If the [Firewall portion](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-firewall) of the [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide has been completed, add a line to `/etc/iptables.firewall.rules` allowing traffic to your IRC port. -{{< /note >}} - - - ![ZNC's Web Admin](znc-web-admin.png) - -## Connect to The Client - -### HexChat ### - -You can use any preferred GUI or CLI client to connect to ZNC. For the example below, we'll be using [HexChat](https://hexchat.github.io/index.html). - -1. Open HexChat, add your desired nicknames, and then create a new network. In this example, the network is called **ZNCserver**: - - ![ZNC](znc-hexchat-1.png) - -2. With **ZNCserver** selected, click `Edit...`. - -3. Add your server's IP address and port to the list. If not using a signed certificate, select *Accept invalid SSL certificated*. Input your password: - - ![ZNC](znc-hexchat-2.png) - - Close the window when done. - -3. Press **Connect**. You should be connected to your ZNC server and from there to any networks and channels you've configured to autojoin. - -### Konversation ### - -1. Open Konversation, click 'New...' - - ![ZNC](znc-konversation-1.png) - -2. Enter a name for the new network. For this example the network is **linode-znc**. Then click 'Add...' to open the dialog to add the server. - - ![ZNC](znc-konversation-2.png) - -3. Now enter your network details such as IP Address, Port number, and password. - - ![ZNC](znc-konversation-3.png) - - -## SSL Encryption with a Signed Certificate (Optional) - -If you would like to use a signed certificate to encrypt your connection to ZNC, you can do so by adding your key and certificate to the `znc.pem` file: - - cat domain.key domain.crt > znc.pem diff --git a/docs/guides/applications/project-management/install-farmos/index.md b/docs/guides/applications/project-management/install-farmos/index.md deleted file mode 100644 index 0e919f435a8..00000000000 --- a/docs/guides/applications/project-management/install-farmos/index.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -slug: install-farmos -title: 'How to Install farmOS - a Farm Recordkeeping Application' -description: 'This guide shows you how to prepare a system for, then install and set up the agribusiness management web app, farmOS.' -authors: ["Angel Guarisma"] -contributors: ["Angel Guarisma"] -published: 2017-09-09 -modified: 2021-06-22 -keywords: ["farmos", "Drupal", "LAMP"] -tags: ["drupal", "lamp", "cms"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- - -![Farm_OS Banner](FarmOS.png) - -## What is farmOS? - -[farmOS](http://farmos.org/) is a one-of-a-kind web application that enables farmers to manage and track all aspects of their farm. Built atop Drupal and licensed under [GPL V.3](https://www.gnu.org/licenses/gpl-3.0.en.html), farmOS is a great free-software solution for farms to explore. - -This guide explains how to install, setup and host your own farmOS web app on a Linode using Ubuntu 20.04. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for [setting your Linode's hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and [timezone](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#set-the-timezone). - -1. Follow our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to [create a standard user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account), [harden SSH access](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#harden-ssh-access), and [create firewall rules](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-firewall) for your web server; you may need to make additional firewall exceptions for your specific application. - - {{% content "limited-user-note-shortguide" %}} - -1. Install and configure a [LAMP stack on Ubuntu 20.04](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-20-04). Skip the configuration steps for setting up MySQL and use the steps outlined in this guide instead. - -## MySQL Setup - -1. Configure your database for PHP. When prompted, choose the `apache2` web server for automatic configuration, `Yes` to automatically configure a database, then enter and confirm your database root password: - - sudo apt install php-mysql phpmyadmin - -1. Log in to MariaDB with your database root password: - - mysql -u root -p - -1. Create a database and a database user with necessary privileges, replacing `secure_password` with a password of your choice: - - CREATE DATABASE farmdb; - CREATE USER 'farm_user'@'localhost' IDENTIFIED BY 'secure_password'; - GRANT ALL PRIVILEGES ON farmdb.* TO 'farm_user'@'localhost'; - FLUSH PRIVILEGES; - - -1. Exit MariaDB: - - quit - -## Download and Install farmOS - -1. Navigate to your site's document root. If you installed and configured your Apache server using our [LAMP stack on Ubuntu 20.04](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-20-04) guide, your document root should be located in the `/var/www/html/example.com/public_html/` directory. Replace `example.com` with your own document root path's name. - - cd /var/www/html/example.com - -1. Download the farmOS tarball. As of writing this guide, farmOS 7.x-1.7 is the latest version. See [Drupal's download page](https://www.drupal.org/project/farm) for their latest core tarball. - - sudo wget https://ftp.drupal.org/files/projects/farm-7.x-1.7-core.tar.gz - - {{< note type="alert" respectIndent=false >}} -Ensure that the version number matches the farmOS version you wish to download. -{{< /note >}} - -1. Extract the downloaded tarball's contents into your site's document root: - - sudo tar -zxvf farm-7.x-1.7-core.tar.gz -C public_html --strip-components=1 - -1. farmOS depends on a PHP graphics library called GD. Install GD and other dependencies: - - sudo apt install php-gd php-xml php-xmlrpc - -## Configure Apache 2.4 - -1. Enable Apache's [rewrite module](https://httpd.apache.org/docs/current/mod/mod_rewrite.html). This module is necessary since farmOS enables [Clean URLs](https://www.drupal.org/getting-started/clean-urls) by default. - - sudo a2enmod rewrite - -1. Specify the rewrite conditions for your farmOS site's document root in Apache's configuration file using the text editor of your choice. If you installed and configured your Apache server using [LAMP stack on Ubuntu 20.04](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-20-04) guide, the configuration file for your site is located at `/etc/apache2/sites-available/example.com.conf`. - - {{< file "/etc/apache2/sites-available/example.com.conf" conf >}} - - Options Indexes FollowSymLinks - AllowOverride All - Require all granted - RewriteEngine on - RewriteBase / - RewriteCond %{REQUEST_FILENAME} !-f - RewriteCond %{REQUEST_FILENAME} !-d - RewriteRule ^(.*)$ index.php?q=$1 [L,QSA] - -{{< /file >}} - -1. Change the ownership of your site's document root from `root` to `www-data`. This allows you to install modules and themes, and to update Drupal, without being prompted for FTP credentials. - - sudo chown -R www-data:www-data /var/www/html/example.com - -1. Restart Apache so all changes are applied. - - sudo systemctl restart apache2 - -## Configure farmOS - -1. Go to your Linode's domain or [IP address](https://techdocs.akamai.com/cloud-computing/docs/managing-ip-addresses-on-a-compute-instance) in a web browser. This shows you the first step of the farmOS/Drupal web configuration. - -1. The first screen you encounter asks you to choose a profile and a language: - - ![farmOS first screen](firstscreen.png) - -1. Drupal checks if the installation is correct in the **Verify requirements** section. Then, it will move to configuring the database. In this section you should input the information from the database built earlier in this tutorial: - - ![Database Configuration](second.png) - -1. Once farmOS hooks into the database, configure your farmOS site. This is where you define the name and the main user account: - - ![Configure](configure.png) - -1. The next section asks you what modules you want to install. You can install and uninstall modules at any time, but this is a chance to install personalized modules that work for your specific type of farm. - - ![modules](modules.png) - -1. Finally, after installing the modules, you are dropped into the farmOS dashboard: - - ![welcome](welcome.png) - -1. After the installation has finished, you may want to reset your file permissions to avoid security vulnerabilities from your site's document root. If you installed and configured your Apache server using our [LAMP stack on Ubuntu 20.04](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-20-04) guide, your document root should be located in the `/var/www/html/example.com/public_html/` directory: - - sudo chmod 644 sites/default - sudo chmod 644 ./sites/default/settings.php - -## Add Users - -To add users to your farmOS distribution, you can do so from the **People** tab under **Manage**. - - ![Add user](Adduser.png) - -After each user is created, use the **People** tab to verify success: - - ![People screen](peoplescreen.png) - -## Next Steps - -### Registering a Domain Name for farmOS - -To register a domain name (e.g., `yourfarm.com`), check out our guide on the [DNS Manager](https://techdocs.akamai.com/cloud-computing/docs/dns-manager) and add your FQDN (e.g., `farmos.yourfarm.com`) to the Linode Manager. A FQDN provides you, and the people who plan on using farmOS, the ability to navigate to a URL instead of your Linode's public IP address. If you plan on using farmOS internally, you can skip this step. - -### Generate a Google API Key - -farmOS can interface with GoogleMaps. You need a GoogleAPI key to use this feature. The farmOS official documentation has a section about using GoogleMaps in its [docs](https://farmos.org/hosting/apikeys/). Interfacing with GoogleMaps allows you to save certain geographical areas into farmOS. When creating farmOS projects and tasks, you can use the Google Maps API to pinpoint where the task takes place. diff --git a/docs/guides/applications/remote-desktop/remote-cloud-desktop-using-apache-guacamole/index.md b/docs/guides/applications/remote-desktop/remote-cloud-desktop-using-apache-guacamole/index.md deleted file mode 100644 index 4872e7c3e8a..00000000000 --- a/docs/guides/applications/remote-desktop/remote-cloud-desktop-using-apache-guacamole/index.md +++ /dev/null @@ -1,134 +0,0 @@ ---- -slug: remote-cloud-desktop-using-apache-guacamole -title: "Using a Cloud Desktop on a Browser with Apache Guacamole" -description: "Instruction on using Apache Guacamole to access a remote server's virtual desktop from a browser." -authors: ["Matt Wildman","Linode"] -contributors: ["Matt Wildman","Linode"] -published: 2017-11-17 -modified: 2021-09-08 -keywords: ["remote desktop", "Apache Guacamole", "TeamViewer", "VNC", "Chrome OS", "xfce", "unity"] -tags: ["docker", "mysql"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Apache Guacamole](https://guacamole.incubator.apache.org/)' - - '[Apache Tomcat](https://tomcat.apache.org/)' -aliases: [] ---- - -![Virtual Cloud Desktop Using Apache Guacamole](Apache_Guacamole.jpg) - -Apache Guacamole is an HTML5 application useful for accessing a remote desktop through RDP, VNC, and other protocols. You can create a virtual cloud desktop where applications can be accessed through a web browser. This guide will cover the installation of Apache Guacamole through Docker, then access a remote desktop environment hosted on a Linode. - -## Install Apache Guacamole - -Apache Guacamole can be installed and configured on a Linode Compute Instance using one of the following methods: - -1. **Akamai Quick Deploy Apps:** Deploy the [Apache Guacamole App](https://www.linode.com/marketplace/apps/linode/apache-guacamole/) through Akamai Quick Deploy Apps to automatically install Guacamole, VNC software, and a desktop environment. This is the easiest method and enables you to quickly get up and running without needing to install and configure everything manually. Just note, when choosing this method you are limited to the Distribution Images supported by the Quick Deploy App. - -1. **Docker:** Alternatively, you can deploy Apache Guacamole's Docker images and manually configure the software yourself. This method strikes a balance between ease of installation and custom configuration. It may be more advanced, but provides you with greater control over your environment and configuration. See [Installing Apache Guacamole through Docker](/cloud/guides/installing-apache-guacamole-through-docker) for instructions. - -1. **Natively:** For maximum control over every step of the installation process, Apache Guacamole can be manually installed on your system from source. This is the most advanced method. If choosing this method, review [Installing Apache Guacamole on Ubuntu and Debian](/cloud/guides/installing-apache-guacamole-on-ubuntu-and-debian) and then return to this guide. You can also see [Installing Guacamole natively](https://guacamole.apache.org/doc/gug/installing-guacamole.html) on the official documentation for additional instructions. - -## Setting Up VNC and a Desktop Environment - -Before you're able to remotely connect to your server's desktop, a desktop environment and VNC server must be installed. - -1. **Install your preferred desktop environment**, such as Xfce, Gnome, KDE, or Unity. A desktop environment is the GUI used to interact with the system, typically through cascading windows (such as Windows and macOS). Most Linux server-based distributions (like those for deployment on Linode) do not come with one pre-installed. The installation and configuration for a desktop environment depends on the environment you chose and the distribution you are running. The following instructions should work on modern Ubuntu distributions: - - **Xfce:** This is a lightweight desktop environment that's a good choice for users of 1GB or 2GB Compute Instances. - - sudo apt install xfce4 xfce4-goodies - - **Unity:** The default feature-rich desktop environment for modern Ubuntu distributions. Unity is more resource intensive and requires a Compute Instance with 4GB of memory or more. - - sudo apt install --no-install-recommends ubuntu-desktop gnome-panel gnome-settings-daemon metacity nautilus gnome-terminal - -1. **Install a VNC server.** This guide uses tightvncserver but other options (such as tigervnc, x11vnc, and vino) should also work. See the "Which VNC Server?" section on the [Configuring Guacamole](https://guacamole.apache.org/doc/0.9.1/gug/configuring-guacamole.html) guide if you'd like to explore other software. - - sudo apt install tightvncserver - -1. Start the VNC server. This prompts you for a secure password that's no longer than 8 characters. - - vncserver - - For setups requiring more security, deploying Guacamole as a [reverse proxy with SSL encryption is highly recommended](https://guacamole.incubator.apache.org/doc/gug/proxying-guacamole.html). - -1. Ensure to start the desktop environment with `.vnc/xstartup` otherwise only a gray screen will be displayed. - - echo 'startxfce4 &' | tee -a .vnc/xstartup - - Alternate Unity configuration example: - - {{< file "~/.vnc/xstartup" bash >}} -#!/bin/sh - -xrdb $HOME/.Xresources -xsetroot -solid grey -#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" & -#x-window-manager & -# Fix to make GNOME work -export XKL_XMODMAP_DISABLE=1 -/etc/X11/Xsession - -gnome-panel & -gnome-settings-daemon & -metacity & -nautilus & -{{< / file >}} - -## Opening Guacamole in a Browser - -1. Before connecting to the VNC server, create an SSH tunnel replacing `user` and `example.com` with the Linode's user and public IP. - - ssh -L 5901:localhost:5901 -N -f -l user example.com - -1. Connect to the VNC server and if `example-guacamole`, `example-guacd`, and `example-mysql` are all running, navigate to `localhost:8080/guacamole/`. The default login credentials are `guacadmin` and password `guacadmin`. This should be changed as soon as possible. - - ![Guacamole Login](guac_login.png) - -{{< note >}} -After you connect to the VNC server if you are not able to open the default browser such as the `Debian Sensible Browser` and see `Failed to Execute Default Web Browser. Input/Output error`, then try installing a different browser such as `FireFox` or others. -{{< /note >}} - -## New Connection in Guacamole - -VNC, RDP, SSH, and Telnet are supported. This section of the guide will show how to navigate the browser interface and add a new connection. - -1. In the Guacamole dashboard, click the top right drop down menu and select *Settings*. Under *Connections*, press the *New Connection* button. - - ![Guacamole Settings](guac_settings.png) - -1. Under **Edit Connection**, choose a name. Under **Parameters**, the hostname is the public IP of the Linode. The port is 5900 plus the display number - in this case, port 5901. Enter the 8 character password. - - ![Guacamole VNC Configuration](guac_vnc_config.png) - - The [official documentation](https://guacamole.incubator.apache.org/doc/gug/configuring-guacamole.html#vnc) has detailed descriptions of all parameter names. - - {{< note respectIndent=false >}} -If you have multiple displays running on the same Linode, increment the port number for each display: 5902, 5903, etc. If your remote displays are hosted on different Linodes, each display should still use port 5901. -{{< / note >}} - -1. From the top right drop down menu, click *Home*. The new connection is now available. - - **CTRL** + **ALT** + **SHIFT** - Opens menu for clipboard, keyboard/mouse settings, and the navigation menu. - - ![Guacamole Drop Down](guac_menu.png) - -1. Press back on the browser to return to the *Home* menu. - -1. Additional connections can be made, and simultaneous connections can be made in new browser tabs. - - ![Guacamole Recent Connections](guac_recent.png) - -This guide aimed to streamline the installation process and demonstrate remote desktop with Apache Guacamole as quickly as possible. There are many features such as screen recording, two factor authentication with Duo, file transfer via SFTP, and much more. As an Apache Incubator project, expect to see further developments in the near future. - -## Is Apache Guacamole Secure? - -There have been certain well-known security vulnerabilities associated with Apache Guacamole, though (at the time of this writing) they have been patched and updates have been released. Most notable is the one identified commonly by security researchers around Reverse RDP vulnerabilities. Any Guacamole version released before January 2020 is vulnerable to reverse RDP and multiple other vulnerabilities in FreeRDP. - -**If you are using an old version of Apache Guacamole released before January 2020, consider upgrading your Apache Guacamole to a newer, stable, and secure version.** - -With the newer versions, the following vulnerabilities have recently surfaced and patched by FreeRDP and Apache Foundation: - -1. **Reverse attack to take control of gateway:** With this, any compromised system or machine inside of the network can exploit an incoming connection and compromise the gateway. This enables the compromised system or machine to take over the gateway. -2. **An internal attack to compromise the gateway:** When an attacker is an internal member of the organization(commonly an employee or a contractor with access to the network), they can use these exploits to gain full access to the gateway. diff --git a/docs/guides/applications/remote-desktop/running-graphic-software-xforwarding-debian/index.md b/docs/guides/applications/remote-desktop/running-graphic-software-xforwarding-debian/index.md deleted file mode 100644 index e3550331deb..00000000000 --- a/docs/guides/applications/remote-desktop/running-graphic-software-xforwarding-debian/index.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -slug: running-graphic-software-xforwarding-debian -title: 'Run Graphic Software on Your Linode with X-Forwarding on Debian' -description: Forward the X11 Server Through SSH to Run GUI Applications from Your Linode -authors: ["Alex Fornuto"] -contributors: ["Alex Fornuto"] -published: 2014-04-10 -modified: 2014-04-25 -keywords: ["x11", "debian", "debian 7", " debian 8", "x-forwarding", "ssh", "x over ssh"] -tags: ["debian", "ssh"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[Xming](http://www.straightrunning.com/XmingNotes/)' - - '[Cygwin/X](http://x.cygwin.com/)' - - '[MobaXterm](http://mobaxterm.mobatek.net/)' - - '[XQuartz](http://xquartz.macosforge.org/)' -relations: - platform: - key: forward-x11-gui - keywords: - - distribution: Debian ---- - -On occasion you may want to run an application that requires a graphic interface from your Linode. By using X forwarding, this is easy to accomplish. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Install X11 on your Linode - -1. Before we begin, make sure your Linode's software is up to date: - - sudo apt-get update - sudo apt-get upgrade - -2. One of the great things about using a Linux distribution with a dependency-aware package manager is that you can just install the application you want to run, and it will make sure you have all the required software. If you're installing a graphic utility, that will include X. For now, let's install `xauth`, which is required for X to authenticate through the SSH session: - - sudo apt-get install xauth - -## Install X11 on the Client - -### Linux - -If you're using a Linux desktop environment on your local PC, you already have X11 running. - -### Mac - -Apple contributes to the development of XQuartz, an X11 server designed to run on OS X. Download the software [here](http://xquartz.macosforge.org/) and follow the installer's instructions. - -### Windows - -There are several pieces of software available that can provide an X server on Windows. We've listed the most notable options below: - -- [Cygwin/X](http://x.cygwin.com/) - Cygwin provides a Unix-like shell to Windows and can support an X server as well. -- [MobaXterm](http://mobaxterm.mobatek.net/) - MobaXterm is an all-in-one tool for accessing remote systems across multiple protocols. -- [Xming](http://www.straightrunning.com/XmingNotes/) - Unlike the previous two, Xming is just the X server itself. It is designed to be a lean standalone system and touts portability. - -It's up to you to choose the software that best suits your needs. - -## Connect - -Connecting from a Linux or OS X client requires adding one extra parameter to your SSH command, the `-X` flag: - - ssh -X root@12.34.56.78 - -Connecting from a Windows machine will depend on what software you have chosen. Please refer to the software documentation for instructions on initiating an SSH connection with X forwarding. - -## Test - -1. First lets install a simple graphic application to test with: - - sudo apt-get install x11-apps - -2. Run the following command to launch a program that watches your cursor: - - xeyes - -3. Now, let's try something a bit bigger. Iceweasel is the name given to Debian's version of Mozilla's Firefox browser. The following commands will install and launch Iceweasel from your Linode: - - sudo apt-get install iceweasel - iceweasel & - - ![Iceweasel, running on a Linode](1633-xforwarding_3.png) - -{{< note >}} -More intense graphic software may lag when running in this fashion. Using a Linode in a data center geographically close to you can help reduce latency. -{{< /note >}} \ No newline at end of file diff --git a/docs/guides/applications/voip/install-asterisk-on-centos-7/index.md b/docs/guides/applications/voip/install-asterisk-on-centos-7/index.md deleted file mode 100644 index 33d9be445ee..00000000000 --- a/docs/guides/applications/voip/install-asterisk-on-centos-7/index.md +++ /dev/null @@ -1,339 +0,0 @@ ---- -slug: install-asterisk-on-centos-7 -title: 'How to Install Asterisk on CentOS 7' -description: 'This guide shows how to install Asterisk, the open-source private branch exchange (PBX) application for running your own VoIP services, on CentOS 7.' -authors: ["Nick Rahl"] -contributors: ["Nick Rahl"] -published: 2015-09-30 -modified: 2020-12-03 -keywords: ["asterisk 13", "centos 7", "centos", "open source", "private branch exchange", "pbx", "asterisk pbx", "sip", "session initiation protocol", "sip protocol", "IP PBX systems", "VoIP gateways"] -tags: ["centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -dedicated_cpu_link: true -relations: - platform: - key: asterisk-freepbx-telephone - keywords: - - distribution: CentOS 7 -aliases: [] ---- - -![How to Install Asterisk on CentOS 7](how-to-install-asterisk-on-centos-7.jpg "How to Install Asterisk on CentOS 7") - -## What is Asterisk? - -Asterisk is an open source *private branch exchange* (PBX) server that uses *Session Initiation Protocol* (SIP) to route and manage telephone calls. Notable features include customer service queues, music on hold, conference calling, and call recording, among others. - -This guide covers the steps necessary to provision a new CentOS 7 Linode as a dedicated Asterisk server for your home or office. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Before You Begin - -1. Create a CentOS 7 Linode in your closest data center. A 2GB Linode is enough to handle 10-20 concurrent calls using a non-compressed codec, depending on the processing required on each channel. - -1. Ensure you have followed the [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guides to prepare your Linode. **Do not** complete the steps to set up a firewall. - -1. Update your system: - - sudo yum update - -1. Disable SELinux and reboot your Linode. If you have [Lassie](https://techdocs.akamai.com/cloud-computing/docs/recover-from-unexpected-shutdowns-with-lassie) enabled, your Linode is back up and running in a few minutes. - - sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config - - sudo systemctl reboot - -## Configure firewalld - -1. CentOS 7 enables firewalld's `public` zone for the default interface (`eth0`). SSH and DHCPv6 services are also enabled by default. To verify your current firewalld zone: - - sudo firewall-cmd --get-active-zones - sudo firewall-cmd --permanent --list-services - - That should return: - - {{< output >}} -[user@asterisk ~]$ sudo firewall-cmd --get-active-zones -public - interfaces: eth0 -{{< /output >}} - - And: - - {{< output >}} -[user@asterisk ~]$ sudo firewall-cmd --permanent --list-services -ssh dhcpv6-client -{{< /output >}} - -1. Add the SIP services. - - {{< note respectIndent=false >}} -All the following firewalld rules contain the `--permanent` flag to ensure the rules persist after a system reboot. -{{< /note >}} - - sudo firewall-cmd --zone=public --permanent --add-service={sip,sips} - -1. Depending on your needs, you may want to add other related ports: - - - MGCP - If you use media gateway control protocol in your configuration. - - sudo firewall-cmd --zone=public --permanent --add-port=2727/udp - - - RTP - The media stream - you can change this in `/etc/asterisk/rtp.conf`. - - sudo firewall-cmd --zone=public --permanent --add-port=10000-20000/udp - - - If you plan to use FreePBX to manage Asterisk, add the following rule: - - sudo firewall-cmd --zone=public --permanent --add-service={http,https} - - - IAX - If you need IAX, add the following rule. IAX is "Inter-Asterisk Exchange" and was meant to allow multiple Asterisk servers to communicate with one another. Some VOIP trunking providers use this, but most use SIP. Unless your VOIP provider requires it or you are running multiple Asterisk servers, you probably don't need IAX or IAX2. - - sudo firewall-cmd --zone=public --permanent --add-port=4569/udp - - -1. Verify your new configuration with: - - sudo firewall-cmd --permanent --list-services - sudo firewall-cmd --permanent --list-ports - - You should see the services and ports you just added in addition to default SSH and DCHPv6 services: - - {{< output >}} -[user@asterisk ~]$ sudo firewall-cmd --list-ports -2727/udp 10000-20000/udp 4569/udp -{{< /output >}} - - {{< output >}} -[user@asterisk ~]$ sudo firewall-cmd --permanent --list-services -ssh dhcpv6-client sip sips http https -{{< /output >}} - - -## Install PJPROJECT - -PJPROJECT is Asterisk's SIP channel driver. It should improve call clarity and performance over older drivers. - -1. Install build dependencies: - - sudo yum install epel-release gcc-c++ ncurses-devel libxml2-devel wget openssl-devel newt-devel kernel-devel-`uname -r` sqlite-devel libuuid-devel gtk2-devel jansson-devel binutils-devel bzip2 patch libedit libedit-devel - -1. As **a non-root user**, create a working directory for the build: - - mkdir ~/build-asterisk - -1. Change to that directory: - - cd ~/build-asterisk - -1. Use `wget` to download the PJSIP driver source code: - - wget https://www.pjsip.org/release/2.8/pjproject-2.8.tar.bz2 - -1. Extract it: - - tar -jxvf pjproject-2.8.tar.bz2 - -1. Change to the newly created directory: - - cd pjproject-2.8 - -1. Specify the compiling flags and options: - - ./configure CFLAGS="-DNDEBUG -DPJ_HAS_IPV6=1" --prefix=/usr --libdir=/usr/lib64 --enable-shared --disable-video --disable-sound --disable-opencore-amr - -1. Ensure that all dependencies are in place: - - make dep - -1. If `make dep` completes successfully, then build the plugin. It should only take a few minutes. - - make - -1. Install the packages: - - sudo make install - sudo ldconfig - -1. Ensure the libraries have been properly installed: - - sudo ldconfig -p | grep pj - - You should see: - - {{< output >}} - libpjsua2.so.2 (libc6,x86-64) => /lib64/libpjsua2.so.2 - libpjsua2.so (libc6,x86-64) => /lib64/libpjsua2.so - libpjsua.so.2 (libc6,x86-64) => /lib64/libpjsua.so.2 - libpjsua.so (libc6,x86-64) => /lib64/libpjsua.so - libpjsip.so.2 (libc6,x86-64) => /lib64/libpjsip.so.2 - libpjsip.so (libc6,x86-64) => /lib64/libpjsip.so - libpjsip-ua.so.2 (libc6,x86-64) => /lib64/libpjsip-ua.so.2 - libpjsip-ua.so (libc6,x86-64) => /lib64/libpjsip-ua.so - libpjsip-simple.so.2 (libc6,x86-64) => /lib64/libpjsip-simple.so.2 - libpjsip-simple.so (libc6,x86-64) => /lib64/libpjsip-simple.so - libpjnath.so.2 (libc6,x86-64) => /lib64/libpjnath.so.2 - libpjnath.so (libc6,x86-64) => /lib64/libpjnath.so - libpjmedia.so.2 (libc6,x86-64) => /lib64/libpjmedia.so.2 - libpjmedia.so (libc6,x86-64) => /lib64/libpjmedia.so - libpjmedia-videodev.so.2 (libc6,x86-64) => /lib64/libpjmedia-videodev.so.2 - libpjmedia-videodev.so (libc6,x86-64) => /lib64/libpjmedia-videodev.so - libpjmedia-codec.so.2 (libc6,x86-64) => /lib64/libpjmedia-codec.so.2 - libpjmedia-codec.so (libc6,x86-64) => /lib64/libpjmedia-codec.so - libpjmedia-audiodev.so.2 (libc6,x86-64) => /lib64/libpjmedia-audiodev.so.2 - libpjmedia-audiodev.so (libc6,x86-64) => /lib64/libpjmedia-audiodev.so - libpjlib-util.so.2 (libc6,x86-64) => /lib64/libpjlib-util.so.2 - libpjlib-util.so (libc6,x86-64) => /lib64/libpjlib-util.so - libpj.so.2 (libc6,x86-64) => /lib64/libpj.so.2 - libpj.so (libc6,x86-64) => /lib64/libpj.so -{{< /output >}} - - - -## Install Asterisk - -1. Return to your build directory: - - cd ~/build-asterisk - -1. Download the latest version of Asterisk 16: - - wget http://downloads.asterisk.org/pub/telephony/asterisk/asterisk-16-current.tar.gz - -1. Untar the file: - - tar -zxvf asterisk-16-current.tar.gz - -1. Switch to the new Asterisk directory, replacing `16.1.1` if needed: - - cd asterisk-16.1.1 - - -### Enable MP3 Support - -1. To use MP3 files for Music on Hold, install Subversion: - - sudo yum install svn - -1. Run the configuration script: - - contrib/scripts/get_mp3_source.sh - - -### Configure and Build Asterisk - -1. In your build directory for Asterisk, run the `configure` script to prepare the Asterisk source code for compiling: - - ./configure --libdir=/usr/lib64 --with-jansson-bundled - -1. Start the build process. After a short while, you should see a menu on screen allowing you to configure the features you want to build. This also produces generic binaries instead of native architecture optimized binaries. - - make menuselect --disable BUILD_NATIVE menuselect.makeopts - -1. If you want to use the MP3 format with Music on Hold, you should select `Add-Ons`, then use the right arrow to move to the right-hand list. Navigate to `format_mp3` and press **Enter** to select it. - -1. Select additional core sound packages and Music on Hold packages in the left menu, and enable `.wav` format for your desired language (ie. use the `EN` package for English.). - -1. Press **F12** to save and exit. - -1. Compile Asterisk. When finished, you should see a message which says Asterisk has successfully been built. - - sudo make - -1. Install Asterisk: - - sudo make install - -1. Install sample configuration files: - - sudo make samples - -1. Configure Asterisk to start itself automatically on boot: - - sudo make config - - -### Test Connection - -You now have a working Asterisk phone server. Fire up Asterisk and make sure it runs. - -1. Start Asterisk: - - sudo systemctl start asterisk - -1. To ensure that asterisk service starts even after a reboot, enable the service: - - sudo systemctl enable asterisk - -1. Connect to Asterisk: - - sudo asterisk -rvv - - You should see an output similar to the following: - - {{< output >}} -Asterisk 16.0.0, Copyright (C) 1999 - 2018, Digium, Inc. and others. -Created by Mark Spencer -Asterisk comes with ABSOLUTELY NO WARRANTY; type 'core show warranty' for details. -This is free software, with components licensed under the GNU General Public -License version 2 and other licenses; you are welcome to redistribute it under -certain conditions. Type 'core show license' for details. -========================================================================= -Connected to Asterisk 16.0.0 currently running on li73-122 (pid = 980) -{{< /output >}} - -1. To see a list of possible commands: - - core show help - -1. To disconnect type: - - exit - - Once disconnected, Asterisk continues to run in the background. - -## Next Steps - -Now that you have an Asterisk server running on your Linode, it's time to connect some phones, add extensions, and configure the various options that are available with Asterisk. For detailed instructions, check out the Asterisk Project's guide to [Configuring Asterisk](https://wiki.asterisk.org/wiki/display/AST/Basic+PBX+Functionality). - -{{< note type="alert" >}} -When running a phone system on a remote server such as a Linode, it's always good practice to secure the signaling data with TLS and the audio portion of calls using SRTP to prevent eavesdropping. Once you have a working dial-plan, be sure to follow the [Secure Calling Guide](https://wiki.asterisk.org/wiki/display/AST/Secure+Calling) to encrypt your communications. -{{< /note >}} diff --git a/docs/guides/databases/cassandra/how-to-install-apache-cassandra-on-centos-7/index.md b/docs/guides/databases/cassandra/how-to-install-apache-cassandra-on-centos-7/index.md deleted file mode 100644 index 75d2ea8ef58..00000000000 --- a/docs/guides/databases/cassandra/how-to-install-apache-cassandra-on-centos-7/index.md +++ /dev/null @@ -1,256 +0,0 @@ ---- -slug: how-to-install-apache-cassandra-on-centos-7 -title: "Installing Apache Cassandra on CentOS 7" -title_meta: "How to Install Apache Cassandra on CentOS 7" -description: 'This guide will show you how to deploy a scalable and development-driven NoSQL database with Apache Cassandra on a Linode running CentOS 7.' -authors: ["Andrew Lescher"] -contributors: ["Andrew Lescher"] -published: 2017-06-12 -modified: 2022-05-16 -keywords: ["cassandra", " apache cassandra", " centos 7", " ubuntu 18.04", " database", " nosql"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -relations: - platform: - key: install-apache-cassandra - keywords: - - distribution: CentOS 7 -external_resources: - - '[Cassandra Documentation](http://cassandra.apache.org/doc/latest/)' - - '[Cassandra cqlshrc File Configuration Overview](http://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlshUsingCqlshrc.html)' - - '[Cassandra .yaml Configuration File Overview](http://cassandra.apache.org/doc/latest/configuration/cassandra_config_file.html)' - - '[Recommended Production Settings For Apache Cassandra](http://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettings.html)' - - '[The Cassandra Query Language (CQL)](http://cassandra.apache.org/doc/latest/cql/index.html)' -tags: ["centos","database","nosql"] -image: Apache_Cassandra.png ---- - -## Introduction to Apache Cassandra - -The Cassandra NoSQL database is ideal for situations that require maximum data redundancy and uptime, ease of horizontal scaling across multiple unique servers, and rapidly evolving project demands during the development life cycle which would otherwise be heavily restricted by traditional relational database implementations. Apache Cassandra is an open-source application that is managed in a simple command line interface using the Cassandra Query Language, or CQL. CQL is syntactically similar to the Structured Query Language, making it easy to pick up for those already familiar with SQL. - -After completing this guide, you will have a single-node, production-ready installation of Apache Cassandra hosted on your Linode. This tutorial will cover basic configuration options, as well as harden database security. In order to successfully execute the commands in this guide, you will need to run them as the "root" user or log in using an account with root privileges, prefixing each command with `sudo`. - -## Install Cassandra - -### Before You Begin - -1. Complete the [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide for setting up a new Linode. -2. While it is recommended you complete the entire [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide, it will be necessary at least to possess a limited user account. - -### Add Repositories and GPG Keys - -1. Install the "yum-utils" package: - - yum install yum-utils -y - -2. Add the Datastax repository: - - yum-config-manager --add-repo http://rpm.datastax.com/community - -4. Add the public key for the datastax repository. Create a directory for the downloaded key: - - mkdir ~/.keys - -5. Navigate to the ".keys" directory you just created and download the public key: - - curl -o repo_key http://rpm.datastax.com/rpm/repo_key - -6. The key should now be contained in a file called `repo_key`. Install the key with the package manager: - - rpm --import repo_key - -## Install Cassandra and Supporting Applications - -Update the system and install Java along with Cassandra. NTP will help keep the Cassandra node synced to the correct time. - -1. Install Cassandra, Java, and NTP: - - yum update && yum upgrade - yum install java dsc30 cassandra30-tools ntp - -## Activate Cassandra - -1. Enable Cassandra on system boot and verify that it is running: - - systemctl enable cassandra - systemctl start cassandra - systemctl -l status cassandra - -3. Check the status of the Cassandra cluster: - - nodetool status - - If `UN` is displayed in the output, the cluster is working. Your output should resemble this: - - {{< output >}} - Status=Up/Down - |/ State=Normal/Leaving/Joining/Moving - -- Address Load Tokens Owns (effective) Host ID Rack - UN 127.0.0.1 103.51 KiB 256 100.0% c43a2db6-8e5f-4b5e-8a83-d9b6764d923d rack1 -{{< /output >}} - - If you receive connection errors, see [Troubleshooting Connection Errors](#troubleshooting-connection-errors). - -## Configure Cassandra - -### Enable Security Features - -1. Enable user login authentication. Make a backup of the Cassandra configuration file "cassandra.yaml." - - {{< note respectIndent=false >}} -The CentOS 7 installation already includes a backup file located at `/etc/cassandra/conf/cassandra.yaml.orig`. -{{< /note >}} - - cp /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.backup - -2. Open "cassandra.yaml" in your preferred text editor: - - vim /etc/cassandra/conf/cassandra.yaml - -3. Match the following variables in the file to the values shown below. If any values are commented out, uncomment them. The rest of the properties found in the cassandra.yaml config file should be set based on your project's particular requirements and how you plan to utilize Cassandra. The default configuration should work well for development. - -More information on this file can be found by following the *Cassandra .yaml Configuration File Overview* link in the "External Resources" section. - -{{< file "/etc/cassandra/conf/cassandra.yaml" yaml >}} -. . . -authenticator: org.apache.cassandra.auth.PasswordAuthenticator -authorizer: org.apache.cassandra.auth.CassandraAuthorizer -role_manager: CassandraRoleManager -roles_validity_in_ms: 0 -permissions_validity_in_ms: 0 -. . . -{{< /file >}} - -After editing the file restart Cassandra. - -## Add An Administration Superuser - -1. Open the Cassandra command terminal by typing `cqlsh`. Log in with the credentials shown below for the default user `cassandra`: - - cqlsh -u cassandra -p cassandra - -1. Create a new superuser. Replace the brackets as well as the content inside with the applicable information: - - cassandra@cqlsh> CREATE ROLE [new_superuser] WITH PASSWORD = '[secure_password]' AND SUPERUSER = true AND LOGIN = true; - -1. Log out by typing `exit`. - -1. Log back in with the new superuser account using the new credentials, and remove the elevated permissions from the Cassandra account: - - superuser@cqlsh> ALTER ROLE cassandra WITH PASSWORD = 'cassandra' AND SUPERUSER = false AND LOGIN = false; - superuser@cqlsh> REVOKE ALL PERMISSIONS ON ALL KEYSPACES FROM cassandra; - -5. Grant all permissions to the new superuser account. Replace the brackets and contents inside with your superuser account username: - - superuser@cqlsh> GRANT ALL PERMISSIONS ON ALL KEYSPACES TO [superuser]; - -6. Log out by typing `exit`. - -## Edit The Console Configuration File - -The *cqlshrc* file holds configuration settings that influence user preferences on how Cassandra performs certain tasks. Before proceeding, switch from the "root" user to your administrative Linux user account (you need sudo privileges for this). - -Since your Cassandra username and password can be stored here in plaintext, this file should only be accessible to your administrative user account, and is designed to be inaccessible to other accounts on your Linux system. Do not set this up as the root user. Caution: Before proceeding, fully evaluate the security risks and consequences to your node cluster before adding the [authentication] section. - -1. Create the file *cqlshrc* using your preferred text editor. If the `~/.cassandra` directory does not exist, create it: - - sudo mkdir ~/.cassandra - sudo vim ~/.cassandra/cqlshrc - -2. Copy any sections below that you wish to add to your configuration. Details for this file can be found by following the "Cassandra cqlshrc File Configuration Overview" link in the "External Resources" section. - - {{< note respectIndent=false >}} -CentOS 7 users can find a sample file containing all the configuration options at `/etc/cassandra/conf/cqlshrc.sample`. -{{< /note >}} - - {{< file "~/.cassandra/cqlshrc" aconf >}} -. . . -;; Options that are common to both COPY TO and COPY FROM - -[copy] -;; The string placeholder for null values -nullval=null -;; For COPY TO, controls whether the first line in the CSV output file will -;; contain the column names. For COPY FROM, specifies whether the first -;; line in the CSV file contains column names. -header=true -;; The string literal format for boolean values -boolstyle = True,False -;; Input login credentials here to automatically login to the Cassandra command line without entering them each time. When this -;; is enabled, just type "cqlsh" to start Cassandra. -[authentication] -username=[superuser] -password=[password] - -;; Uncomment to automatically use a certain keyspace on login -;; keyspace=[keyspace] - -[ui] -color=on -datetimeformat=%Y-%m-%d %H:%M:%S%z -completekey=tab -;; The number of digits displayed after the decimal point -;; (note that increasing this to large numbers can result in unusual values) -float_precision = 5 -;; The encoding used for characters -encoding = utf8 -. . . -{{< /file >}} - - -3. Save and close the file. Update the file and directory with the following permissions: - - sudo chmod 440 ~/.cassandra/cqlshrc - sudo chmod 700 ~/.cassandra - -5. If you enabled the auto-login feature, login by typing `cqlsh`. The command terminal should open, and your superuser name should be visible in the command line. - -## Rename the Cluster - -Update your default cluster name from "Test Cluster" to your desired name. - -1. Login to the control terminal with cqlsh. Replace `[new_name]` with your new cluster name: - - UPDATE system.local SET cluster_name = '[new_name]' WHERE KEY = 'local'; - -2. Edit the cassandra.yaml file and replace the value in the `cluster_name` variable with the new cluster name you just set. - - vim /etc/cassandra/conf/cassandra.yaml - -3. Save and close. - -4. From the Linux terminal (not cqlsh), run `nodetool flush system`. This will clear the system cache and preserve all data in the node. - -5. Restart Cassandra. Log in with cqlsh and verify the new cluster name is visible. - -## Troubleshooting Connection Errors - -If you receive connection errors when running `nodetool status`, you may need to manually enter networking information. - -1. Open the `cassandra-env.sh` file in a text editor. - - sudo vim /etc/cassandra/conf/cassandra-env.sh - -1. Search for `-Djava.rmi.server.hostname=` in the file. Uncomment this line and add your loopback address or public IP address by replacing `` at the end of the line: - - {{< file "/etc/cassandra/conf/cassandra-env.sh" bash >}} -. . . -JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=" -. . . -{{< /file >}} - -1. Restart Cassandra after you've finished updating the `cassandra-env.sh` file: - - sudo systemctl restart cassandra - -1. Check the node status again after the service restarts: - - nodetool status - - {{< note respectIndent=false >}} -It may take a few seconds for Cassandra to refresh the configuration. If you receive another connection error, try waiting 15 seconds before rechecking the node status. -{{< /note >}} - -## Where To Go From Here - -Be sure to check out the links in the "External Resources" section, which will help you further configure Cassandra to your needs, as well as provide resources to improve your understanding and ability to use Cassandra. To fully utilize the capabilities of Cassandra in a production setting, additional nodes should be added to your cluster. See the companion guide, "Deploy Additional Nodes To The Cassandra Cluster" to get started. diff --git a/docs/guides/databases/cassandra/how-to-install-apache-cassandra-on-debian-9/index.md b/docs/guides/databases/cassandra/how-to-install-apache-cassandra-on-debian-9/index.md deleted file mode 100644 index 819d1e8a9e6..00000000000 --- a/docs/guides/databases/cassandra/how-to-install-apache-cassandra-on-debian-9/index.md +++ /dev/null @@ -1,313 +0,0 @@ ---- -slug: how-to-install-apache-cassandra-on-debian-9 -title: "Install Apache Cassandra on Debian 9" -title_meta: "How to Install Apache Cassandra on Debian 9" -description: 'This guide presents instructions to deploy a scalable and development-driven NoSQL database with Apache Cassandra for Debian 9.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-01-30 -keywords: ["cassandra", " apache cassandra", " centos 7", "Debian 9", " database", " nosql"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: L_Cass_on_Debian9.png -external_resources: - - '[Cassandra Documentation](http://cassandra.apache.org/doc/latest/)' - - '[Cassandra cqlshrc File Configuration Overview](http://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlshUsingCqlshrc.html)' - - '[Cassandra .yaml Configuration File Overview](http://cassandra.apache.org/doc/latest/configuration/cassandra_config_file.html)' - - '[Recommended Production Settings For Apache Cassandra](http://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettings.html)' - - '[The Cassandra Query Language (CQL)](http://cassandra.apache.org/doc/latest/cql/index.html)' -relations: - platform: - key: install-apache-cassandra - keywords: - - distribution: Debian 9 -tags: ["debian","database","nosql"] -aliases: [] -deprecated: true ---- - -After completing this guide, you will have a single-node, production-ready installation of [Apache Cassandra](http://cassandra.apache.org/) hosted on your Linode running Debian 9. This tutorial will cover basic configuration options, as well as harden database security. - -{{< note >}} -In order to successfully execute the commands in this guide, you will need to run them as the `root` user, or log in using an account with root privileges, prefixing each command with `sudo`. -{{< /note >}} - -## Before You Begin - -1. Complete the [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide for setting up a new Linode. -1. While it is recommended you complete the entire [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide, at minimum, you should [add a limited user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account). - -## Install Cassandra and Supporting Applications - -In this section, you will install package dependencies, Java, Cassandra, and update your Linux system software. - -1. Update your system's software packages: - - sudo apt update - -1. Install the required package dependencies: - - sudo apt install apt-transport-https ca-certificates wget dirmngr gnupg software-properties-common - -1. Import the repository’s GPG key using `wget` and add the AdoptOpenJDK APT repository: - - wget -qO - https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public | sudo apt-key add - - sudo add-apt-repository --yes https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/ - -1. Install Java 8: - - sudo apt update - sudo apt install adoptopenjdk-8-hotspot - -1. Verify the version of Java you just installed: - - java -version - -1. Add Cassandra's GPG keys: - - wget -q -O - https://www.apache.org/dist/cassandra/KEYS | sudo apt-key add - - -1. Add the Cassandra repository to your Debian system's sources list: - - sudo sh -c 'echo "deb http://www.apache.org/dist/cassandra/debian 311x main" > /etc/apt/sources.list.d/cassandra.list' - - {{< note respectIndent=false >}} -You may want to follow the link to the Apache repository to confirm that “40x” is the latest available version. - {{< /note >}} - -1. Update your packages index and install Cassandra: - - sudo apt update - sudo apt install cassandra - -## Activate Cassandra - -1. Enable Cassandra on system boot and verify that it is running: - - sudo systemctl enable cassandra - sudo systemctl start cassandra - sudo systemctl -l status cassandra - -1. Check the status of the Cassandra cluster: - - nodetool status - - If `UN` is displayed in the output, the cluster is working. Your output should resemble the following: - - {{< output >}} -Status=Up/Down -|/ State=Normal/Leaving/Joining/Moving --- Address Load Tokens Owns (effective) Host ID Rack -UN 127.0.0.1 103.51 KiB 256 100.0% c43a2db6-8e5f-4b5e-8a83-d9b6764d923d rack1 - {{< /output >}} - - If you receive connection errors, open the `cassandra-env.sh` file in a text editor. - - sudo vim /etc/cassandra/cassandra-env.sh - - Search for `-Djava.rmi.server.hostname=` in the file. Uncomment this line and add your loopback address or public IP address by replacing `` at the end of the line: - - {{< file "Debian /etc/cassandra/conf/cassandra-env.sh" bash >}} -. . . - -JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=" - -. . . - {{< /file >}} - - - Restart Cassandra after you've finished updating the `cassandra-env.sh` file: - - sudo systemctl restart cassandra - - - Check the node status: - - nodetool status - - {{< note respectIndent=false >}} -It may take a few seconds for Cassandra to refresh the configuration. If you receive another connection error, try waiting 15 seconds before rechecking the node status. - {{< /note >}} - -## Configure Cassandra - -### Enable Security Features - -In this section, you will enable user login authentication. You can also configure other security settings based on your project's needs. - -1. Make a backup of the Cassandra configuration file `cassandra.yaml`. - - sudo cp /etc/cassandra/cassandra.yaml /etc/cassandra/cassandra.yaml.backup - -1. Open `cassandra.yaml` in your preferred text editor: - - {{< note respectIndent=false >}} -Locations of the `cassandra.yaml` file may differ slightly between distros. - {{< /note >}} - - sudo vim /etc/cassandra/cassandra.yaml - -1. Match the following variables in the file to the values shown in the example file. If any values are commented out, uncomment them. The rest of the properties found in the `cassandra.yaml` file should be set based on your project's particular requirements and how you plan to utilize Cassandra. The default configuration should work well for development. - - {{< file "Debian /etc/cassandra/cassandra.yaml" yaml >}} -. . . - -authenticator: org.apache.cassandra.auth.PasswordAuthenticator -authorizer: org.apache.cassandra.auth.CassandraAuthorizer -role_manager: CassandraRoleManager -roles_validity_in_ms: 0 -permissions_validity_in_ms: 0 - -. . . - {{< /file >}} - - More information about this file can be found in the [Cassandra Configuration File](http://cassandra.apache.org/doc/latest/configuration/cassandra_config_file.html) guide in Apache's official documentation. - -1. After editing the configuration file restart Cassandra. - - sudo systemctl restart cassandra - -### Add An Administration Superuser - -1. Open the Cassandra command terminal by typing `cqlsh`. Log in with the credentials shown below for the default user `cassandra`: - - cqlsh -u cassandra -p cassandra - -1. Create a new superuser. Replace the brackets as well as the content inside with the applicable information: - - - CREATE ROLE [new_superuser] WITH PASSWORD = '[secure_password]' AND SUPERUSER = true AND LOGIN = true; - -1. Log out by typing `exit`. - -1. Log back in with the new superuser account and replace the username and password with your new credentials: - - cqlsh -u new-super-user -p my-secure-password - -1. Remove the elevated permissions from the Cassandra account: - - ALTER ROLE cassandra WITH PASSWORD = 'cassandra' AND SUPERUSER = false AND LOGIN = false; - REVOKE ALL PERMISSIONS ON ALL KEYSPACES FROM cassandra; - -1. Grant all permissions to the new superuser account. Replace the brackets and contents inside with your superuser account username: - - GRANT ALL PERMISSIONS ON ALL KEYSPACES TO [superuser]; - -1. Log out by typing `exit`. - -### Edit The Console Configuration File - -The `cqlshrc` file holds configuration settings that influence user preferences and how Cassandra performs certain tasks. - -{{< note >}} -Ensure you complete the steps in this section using your limited user account. This account will need [sudo privileges](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account), if it does not already have them. -{{< /note >}} - -Since your Cassandra username and password can be stored in plaintext, the `cqlshrc` file should only be accessible to your administrative user account, and is designed to be inaccessible to other accounts on your Linux system. - -{{< note type="alert" >}} -Do not complete this section as the root user. Before proceeding, fully evaluate the security risks and consequences to your node cluster before adding the `[authentication]` section. -{{< /note >}} - -1. Create the file `cqlshrc` using your preferred text editor. If the `~/.cassandra` directory does not exist, create it: - - sudo mkdir ~/.cassandra - sudo vim ~/.cassandra/cqlshrc - -1. Copy any sections below that you wish to add to your configuration, and ensure you replace the `superuser` and `password` value in brackets with your own values. Details for this file can be found in the [Configuring cqlsh From a File](https://docs.datastax.com/en/archived/cql/3.3/cql/cql_reference/cqlshUsingCqlshrc.html) guide on the [DataStax](https://www.datastax.com/) site. - - {{< file "~/.cassandra/cqlshrc" aconf >}} -. . . - -;; Options that are common to both COPY TO and COPY FROM - -[copy] -;; The string placeholder for null values -nullval=null -;; For COPY TO, controls whether the first line in the CSV output file will -;; contain the column names. For COPY FROM, specifies whether the first -;; line in the CSV file contains column names. -header=true -;; The string literal format for boolean values -boolstyle = True,False -;; Input login credentials here to automatically login to the Cassandra command line without entering them each time. When this -;; is enabled, just type "cqlsh" to start Cassandra. -[authentication] -username=[superuser] -password=[password] - -;; Uncomment to automatically use a certain keyspace on login -;; keyspace=[keyspace] - -[ui] -color=on -datetimeformat=%Y-%m-%d %H:%M:%S%z -completekey=tab -;; The number of digits displayed after the decimal point -;; (note that increasing this to large numbers can result in unusual values) -float_precision = 5 -;; The encoding used for characters -encoding = utf8 - -. . . -{{< /file >}} - - -1. Save and close the file. - -1. Update the `cqlshrc` file and directory with the following permissions: - - sudo chmod 440 ~/.cassandra/cqlshrc - sudo chmod 700 ~/.cassandra - -1. Login by typing the command below. You will be prompted to enter your password. The `cqlsh` command terminal should open, and your superuser name should be visible in the command line. - - cqlsh -u superuser - - {{< note respectIndent=false >}} -You can also login by providing your username and password: - - cqlsh -u superuser -p password - {{< /note >}} - -## Rename the Cluster - -In this section, you will update your default cluster name from "Test Cluster" to your desired name. - -1. Log into the `cqlsh` control terminal if you are not already logged in. - - cqlsh -u superuser - -1. Replace `[new_name]` with your new cluster name: - - UPDATE system.local SET cluster_name = '[new_name]' WHERE KEY = 'local'; - -1. Type `exit` to return to the Linux command line. - -1. Edit the `cassandra.yaml` file and replace the value in the `cluster_name` variable with the new cluster name you just set. - - sudo vim /etc/cassandra/cassandra.yaml - -1. Save and close. - -1. From the Linux terminal (not cqlsh) clear the system cache. This command will not disturb your node's data. - - nodetool flush system - -1. Restart Cassandra: - - sudo systemctl restart cassandra - -1. Log in with cqlsh and verify the new cluster name is visible. - - cqlsh -u superuser - - {{< output >}} -Connected to my-cluster-name at 127.0.0.1:9042. -[cqlsh 5.0.1 | Cassandra 4.0 | CQL spec 3.4.5 | Native protocol v4] -Use HELP for help. -superuser@cqlsh> - {{}} - -## Where To Go From Here - -Be sure to check out the links in the [More Information](#more-information) section, which will help you further configure Cassandra to your needs, as well as provide resources to improve your understanding and ability to use Cassandra. - -To fully utilize the capabilities of Cassandra in a production setting, additional nodes should be added to your cluster. See the companion guide [Adding Nodes to an Existing Cluster](https://docs.datastax.com/en/archived/cassandra/3.0/cassandra/operations/opsAddNodeToCluster.html) for more information. \ No newline at end of file diff --git a/docs/guides/databases/mariadb/how-to-install-mariadb-on-debian-10/index.md b/docs/guides/databases/mariadb/how-to-install-mariadb-on-debian-10/index.md deleted file mode 100644 index c9e7ca9db5d..00000000000 --- a/docs/guides/databases/mariadb/how-to-install-mariadb-on-debian-10/index.md +++ /dev/null @@ -1,242 +0,0 @@ ---- -slug: how-to-install-mariadb-on-debian-10 -title: "Installing MariaDB on Debian 10" -title_meta: "How to Install MariaDB on Debian 10" -description: "Want to replace MySQL? Read through this guide, which explains how to install MariaDB on Debian 10." -authors: ["Ryan Syracuse"] -contributors: ["Ryan Syracuse"] -published: 2020-01-31 -modified: 2024-05-16 -keywords: ["mariadb", "Debian 10", "debian", "database", "mysql"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -image: Installing_MariaDB_on_Debian10.png -external_resources: - - '[MariaDB Knowledge Base](https://mariadb.com/kb/en)' - - '[MariaDB FAQ](https://mariadb.com/kb/en/mariadb-mariadb-faq/)' - - '[MariaDB SQL commands](https://mariadb.com/kb/en/sql-commands/)' -relations: - platform: - key: how-to-install-mariadb - keywords: - - distribution: Debian 10 -tags: ["debian","mariadb","database"] ---- - -MariaDB is a fork of the popular cross-platform MySQL database management system and is considered a full [drop-in replacement](https://mariadb.com/kb/en/mariadb/mariadb-vs-mysql-features/) for MySQL. MariaDB was created by one of MySQL's original developers in 2009 after MySQL was acquired by Oracle during the Sun Microsystems merger. Today MariaDB is maintained and developed by the [MariaDB Foundation](https://mariadb.org/en/foundation/) and community contributors with the intention of it remaining GNU GPL software. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system and configure your hostname. You may also wish to set the timezone, create a limited user account, and harden SSH access. - - To check your hostname run: - - hostname - hostname -f - - The first command should show your short hostname, and the second should show your fully qualified domain name (FQDN) if you have one assigned. - -## Install and Setup MariaDB - -Install MariaDB using the package manager. - - sudo apt install mariadb-server - -MariaDB will bind to localhost (127.0.0.1) by default. For information on connecting to a remote database using SSH, see our [MySQL remote access guide](/cloud/guides/create-an-ssh-tunnel-for-mysql-remote-access), which also applies to MariaDB. - -{{< note >}} -Allowing unrestricted access to MariaDB on a public IP not advised but you may change the address it listens on by modifying the `bind-address` parameter in `/etc/mysql/my.cnf`. If you decide to bind MariaDB to your public IP, you should implement firewall rules that only allow connections from specific IP addresses. -{{< /note >}} - -### MariaDB Client - -The standard tool for interacting with MariaDB is the `mariadb` client, which installs with the `mariadb-server` package. The MariaDB client is used through a terminal using the `mariadb` command. Review the official [MariaDB Command-Line Client](https://mariadb.com/kb/en/mariadb-command-line-client/) guide for more details on this command. - -### Root Login - -1. Log into MariaDB as the root user: - - sudo mariadb -u root -p - -1. When prompted for login credentials, hit enter. By default MariaDB will authenticate you via the **unix_socket plugin** and credentials are not required. - - You'll then be presented with a welcome header and the MariaDB prompt as shown below: - - {{< output >}} -MariaDB [(none)]> -{{}} - -1. To generate a list of commands for the MariaDB prompt, enter `\h`. You'll then see: - - {{< output >}} -General information about MariaDB can be found at -http://mariadb.org - -List of all MySQL commands: -Note that all text commands must be first on line and end with ';' -? (\?) Synonym for `help'. -clear (\c) Clear the current input statement. -connect (\r) Reconnect to the server. Optional arguments are db and host. -delimiter (\d) Set statement delimiter. -edit (\e) Edit command with $EDITOR. -ego (\G) Send command to mysql server, display result vertically. -exit (\q) Exit mysql. Same as quit. -go (\g) Send command to mysql server. -help (\h) Display this help. -nopager (\n) Disable pager, print to stdout. -notee (\t) Don't write into outfile. -pager (\P) Set PAGER [to_pager]. Print the query results via PAGER. -print (\p) Print current command. -prompt (\R) Change your mysql prompt. -quit (\q) Quit mysql. -rehash (\#) Rebuild completion hash. -source (\.) Execute an SQL script file. Takes a file name as an argument. -status (\s) Get status information from the server. -system (\!) Execute a system shell command. -tee (\T) Set outfile [to_outfile]. Append everything into given outfile. -use (\u) Use another database. Takes database name as argument. -charset (\C) Switch to another charset. Might be needed for processing binlog with multi-byte charsets. -warnings (\W) Show warnings after every statement. -nowarning (\w) Don't show warnings after every statement. - -For server side help, type 'help contents' - -MariaDB [(none)]> -{{}} - -### Securing the Installation - -1. After accessing MariaDB as the root user of your database, enable the **mysql_native_password** -plugin to enable root password authentication: - - USE mysql; - UPDATE user SET plugin='mysql_native_password' WHERE user='root'; - FLUSH PRIVILEGES; - exit; - -1. Run the `mysql_secure_installation` script to address several security concerns in a default MariaDB installation: - - sudo mysql_secure_installation - -You will be given the choice to change the MariaDB root password, remove anonymous user accounts, disable root logins outside of localhost, and remove test databases. It is recommended that you answer `yes` to these options. You can read more about the script in the [MariaDB Knowledge Base](https://mariadb.com/kb/en/mariadb/mysql_secure_installation/). - -## Using MariaDB - -### Create a New MariaDB User and Database - -1. Login to the database again. This time, if you set a password above, enter it at the prompt. - - sudo mariadb -u root -p - -1. In the example below, `testdb` is the name of the database, `testuser` is the user, and `password` is the user's password. You should replace `password` with a secure password: - - CREATE DATABASE testdb; - CREATE user 'testuser'@localhost IDENTIFIED BY 'password'; - GRANT ALL ON testdb.* TO 'testuser' IDENTIFIED BY 'password'; - - You can shorten this process by creating the user *while* assigning database permissions: - - CREATE DATABASE testdb; - GRANT ALL ON testdb.* TO 'testuser' IDENTIFIED BY 'password'; - -1. Then exit MariaDB: - - exit; - -### Create a Sample Table - -1. Log back in as `testuser`, entering the password when prompted: - - sudo mariadb -u testuser -p - -1. Create a sample table called `customers`: - - USE testdb; - CREATE TABLE customers (customer_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, first_name TEXT, last_name TEXT); - - - This creates a table with a `customer_id` field of the type `INT` for integer. - - This field is auto-incremented for new records and used as the primary key. - - Two other fields are created, `first_name` and `last_name` for storing the customer's name. - -1. View the new table: - - SHOW TABLES; - - {{< output >}} -+------------------+ -| Tables_in_testdb | -+------------------+ -| customers | -+------------------+ -1 row in set (0.00 sec) -{{}} - -1. Add some data: - - INSERT INTO customers (first_name, last_name) VALUES ('John', 'Doe'); - -1. View the data: - - SELECT * FROM customers; - - {{< output >}} -+-------------+------------+-----------+ -| customer_id | first_name | last_name | -+-------------+------------+-----------+ -| 1 | John | Doe | -+-------------+------------+-----------+ -1 row in set (0.00 sec) -{{}} - -1. Then exit MariaDB: - - exit; - -## Reset the MariaDB Root Password - -If you forget your root MariaDB password, it can be reset. - -1. Stop the current MariaDB server instance, then restart it with an option to not ask for a password: - - sudo systemctl stop mariadb - -1. Then execute the following command which will allow the database to start without loading the grant tables or networking. - - sudo systemctl set-environment MYSQLD_OPTS="--skip-grant-tables --skip-networking" - -1. Restart MariaDB: - - sudo systemctl start mariadb - -1. Login to the MariaDB server with the root account, this time without supplying a password: - - sudo mariadb -u root - -1. Use the following commands to reset root's password. Replace `password` with a strong password: - - FLUSH PRIVILEGES; - UPDATE mysql.user SET password = PASSWORD('password') WHERE user = 'root'; - -1. Update the authentication methods for the root password: - - UPDATE mysql.user SET authentication_string = '' WHERE user = 'root'; - UPDATE mysql.user SET plugin = '' WHERE user = 'root'; - exit; - -1. Revert the environment settings to allow the database to start with grant tables and networking: - - sudo systemctl unset-environment MYSQLD_OPTS - -1. Then restart MariaDB: - - sudo systemctl start mariadb - -1. You should now be able to log into the database with your new root password: - - sudo mariadb -u root -p \ No newline at end of file diff --git a/docs/guides/databases/mongodb/a-shell-script-to-automatically-backup-mongodb-databases/index.md b/docs/guides/databases/mongodb/a-shell-script-to-automatically-backup-mongodb-databases/index.md deleted file mode 100644 index 91eedf5f5ae..00000000000 --- a/docs/guides/databases/mongodb/a-shell-script-to-automatically-backup-mongodb-databases/index.md +++ /dev/null @@ -1,145 +0,0 @@ ---- -slug: a-shell-script-to-automatically-backup-mongodb-databases -title: "Automatically Backup MongoDB Databases Using Bash Scripting" -description: 'In this guide, you learn how to create a Bash script that creates a backup of all current MongoDB databases. The MongoDB backup is in a .tar file format and is configured to be uploaded to a Linode object storage bucket. You also learn how to configure a Cron job that automatically runs the backup script daily.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2022-04-04 -keywords: ['bash','backup','mongodb','object storage','cron','linux'] -tags: ['database'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[MongoDB](https://www.mongodb.com/)' -- '[Crontab](https://man7.org/linux/man-pages/man5/crontab.5.html)' -- '[Nano](https://www.nano-editor.org/docs.php)' ---- - -MongoDB is a popular non-relationship database management system that stores keys and their values in a collection of documents rather than tables with fixed schemas. MongoDB supports a wide variety of options for horizontal scaling, making it an ideal tool for large enterprise production environments. - -Backing up the data stored in a MongoDB database is an important step to maintain data integrity and disaster recovery plans. To assure that the MongoDB databases are backed up regularly to an external source such as a Linode Object Storage bucket, a simple Bash script can be created. Then, the Bash script can be configured to run daily using the Linux Cron job workflow. -## Before You Begin - -1. Learn about the fundamentals of Linode Object Storage by viewing the [Get Started with Object Storage](https://techdocs.akamai.com/cloud-computing/docs/getting-started-with-object-storage) guide or by reviewing the available [Object Storage guides](https://techdocs.akamai.com/cloud-computing/docs/object-storage). - -1. Create a [Linode Object Storage bucket](https://techdocs.akamai.com/cloud-computing/docs/create-and-manage-buckets). This bucket is used to store your MongoDB backups. - -1. Create a pair of [Access Keys](https://techdocs.akamai.com/cloud-computing/docs/manage-access-keys) for your Linode Object Storage bucket. - -1. Install [MongoDB](/cloud/guides/install-mongodb-on-ubuntu-16-04) on your Linux system. - -{{< note >}} -The steps in this guide are written for a non-root user account. For any commands that require elevated privileges, `sudo` is prefixed at the start of the command syntax. If you’re unfamiliar with the `sudo` command workflow, see the [Linux Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Install Cyberduck CLI - -In this guide, we use the [Cyberduck CLI](https://duck.sh/) for interacting with your Linode Object Storage bucket. Cyberduck CLI, also referred to as [duck](https://duck.sh/), is a command line interface tool that allows you to access, and manage objects stored in your Linode bucket. We use duck for uploading our MongoDB backup files to our Linode bucket. - -If you are using a Debian Linux distribution, install duck with the following commands: - - echo -e "deb https://s3.amazonaws.com/repo.deb.cyberduck.io stable main" | sudo tee /etc/apt/sources.list.d/cyberduck.list > /dev/null - sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys FE7097963FEFBE72 - sudo apt-get update - sudo apt-get install duck - - -If you are using a Red Hat Linux distribution, install [duck](https://duck.sh/) with the following commands: - - echo -e "[duck-stable]\nname=duck-stable\nbaseurl=https://repo.cyberduck.io/stable/\$basearch/\nenabled=1\ngpgcheck=0" | sudo tee /etc/yum.repos.d/duck-stable.repo - sudo yum install duck - -## Create Bash Script - -Navigate into your user's home directory or your desired working directory. This guide uses the `/home/linode-user` directory for example purposes. Create a new file called `backup_mongodb.sh` using the following command: - - touch backup_mongodb.sh - - -Open the above file in the nano text editor and add the following lines of code. For more information on Linux nano, check out the [official nano documentation](https://www.nano-editor.org/docs.php). - - -{{< file "backup_mongodb.sh" sh >}} -#!/bin/bash - -### Variables in this section will need to be edited to reflect your configuration. -###### Start of section - -# Your Linux user's home directory -export HOME=/home/linode-user - -# Your MongoDB's server hostname -HOST=localhost - -# The MongoDB database to be backed up -DBNAME=LINODE_DATABASE - -# Linode Object Storage bucket name. Replace this value with the name of your Linode bucket and it's associated region. This should be in the format {BUCKET_NAME}.{BUCKET_REGION}.linodeobjects.com -BUCKET=mongodb-backups.us-southeast-1.linodeobjects.com - -# Linux user account -USER=linode-user - -# Linode Bucket Access Key. You can use an environment variable to store your Linode Access Key, or you can store it in plain text in this script in the line below. Plain text is not recommended for production environments. -LINODE_ACCESS_KEY=$LINODEACCESSKEY - -###### End of section - -# Current time and date -TIME=`/bin/date +%d-%m-%Y-%T` - -# Directory you'd like the MongoDB backup file to be saved to -DEST=/home/$USER/tmp - -# Command to create a .tar file of the MongoDB backup files -TAR=$DEST/$TIME.tar - -# Command to create the backup directory (-p to avoid warning if the directory already exists) -/bin/mkdir -p $DEST - -# Echo for logging purposes -echo "Backing up $HOST/$DBNAME to Linode $BUCKET on $TIME"; - -# Command to run the mongodump command that dumps all data for the specified database to the backup directory -/usr/bin/mongodump -h $HOST -d $DBNAME -o $DEST - -# Create the .tar file of backup directory -/bin/tar cvf $TAR -C $DEST . - -# Upload the .tar to Linode -duck --username $LINODE_ACCESS_KEY --upload s3://$BUCKET/ $TAR - -# Log the end of the script -echo "Backup of MongoDB databases to Linode bucket $BUCKET completed successfully." - -{{< /file >}} - -Refer to the comments in the code to learn what each line in the code does. - -## Run the Bash Script - -Before setting the above script to run automatically, execute the script to configure your Linode bucket's access key pair. You need to set the appropriate permissions for this script to be executed. For more information on Linux file permissions, review our guide on [Modifying File Permissions with chmod](/cloud/guides/modify-file-permissions-with-chmod). - -Modify the script's permissions, then execute the script with the following commands: - - chmod 755 backup_mongodb.sh - ./backup_mongodb.sh - -Enter your Linode Access keys as prompted, and choose the option to save them. - -{{< note >}} -[Duck](https://duck.sh/) saves your keys in a plain text file at `~.duck/credentials`. It is important to take appropriate measures to secure this file. Saving your keys in this file is necessary for them to be used when this script is run automatically. -{{< /note >}} - -## Schedule Backups as a Cron Job - -Open the Cron task scheduler with the following command: - - crontab -e - -Select your preferred text editor from the menu and enter the following line in your Cron job file: - - 0 9 * * * /path/to/backup_mongodb.sh - -Replace `/path/to/` from the above line with the full directory path to your `backup_mongodb.sh` file. - -The above Cron job is scheduled to run daily at 9:00 AM in your system's configured time zone. For more information on Cron job configuration parameters, review our guide on [Using Cron to Schedule Tasks for Certain Times or Intervals](/cloud/guides/schedule-tasks-with-cron). diff --git a/docs/guides/databases/mysql/how-to-install-mysql-on-centos8/index.md b/docs/guides/databases/mysql/how-to-install-mysql-on-centos8/index.md deleted file mode 100644 index 837f146c3b7..00000000000 --- a/docs/guides/databases/mysql/how-to-install-mysql-on-centos8/index.md +++ /dev/null @@ -1,239 +0,0 @@ ---- -slug: how-to-install-mysql-on-centos8 -title: "How to Install MySQL on CentOS 8" -title_meta: "Installing MySQL on CentOS 8" -description: 'Learn how to install MySQL on CentOS 8 Linux on a cloud server with cloud apps.' -authors: ["John Mueller"] -contributors: ["John Mueller"] -published: 2022-10-11 -keywords: ['Install MySQL on CentOS 8', 'Install MySQL on CentOS Stream 8', 'Install MySQL', 'CentOS 8', 'Connect MySQL to CentOS 8', 'Install MySQL server in Linux'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[MySQL vs MariaDB](https://blog.devart.com/mysql-vs-mariadb.html)' -- '[MariaDB vs MySQL compatibility](https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility/)' -- '[Introduction to Uncomplicated Firewall](https://www.linux.com/training-tutorials/introduction-uncomplicated-firewall-ufw/)' ---- - -## How to Install MySQL on CentOS Stream 8 - -Most business applications require access to data, which in turn makes it important to know how to install and manage a Database Management System (DBMS). There are different types of DBMS, but the most popular is the Relational DBMS (RDBMS) which is based on Structured Query Language (SQL). One of the most popular RDBMS is MySQL. This guide explains how to download, and install MySQL on CentOS Stream 8, and set up important configurations. - - -## The Difference Between MySQL and MariaDB - -Before we get started, understand the relationship between MySQL and MariaDB through their shared history. MySQL is an open-source RDBMS used for everything from small-scale to large-scale industrial applications. Oracle purchased MySQL in May 1995. However, Oracle’s vision of what MySQL should be fell short of some of MySQL developers' and users' expectations. These developers created MariaDB based on the Community Edition of MySQL and released it in October 2009. - -MariaDB is touted as a drop-in replacement for MySQL, but there are differences between the two products. A significant number of features present in MariaDB make the move to the RDBMS a one-way process. Especially, when you plan to use the advanced features without using some sort of special tool to help with the transfer. It also pays to know that [MySQL and MariaDB vary in functionality](https://blog.devart.com/mysql-vs-mariadb.html). For example, MySQL doesn’t support `JSON_EXISTS` or `JSON_QUERY`, and MariaDB lacks support for `JSON_TABLE`. When it comes to SQL support, MySQL provides superior indexing capabilities, while MariaDB supports sequences. The following table provides a quick overview of the significant differences between the two products: - -| Feature | MySQL | MariaDB | -|-----------------------------------------------------------|-------------------------|-------------------------| -| Underlying development languages | C/C++ | C/C++ | -| Maturity | Developed in 1995 so it has a long-term existence. [The server currently has 8K stars and 3.1 forks on GitHub.](https://github.com/mysql/mysql-server) | Developed in 2009 so it’s less mature, especially given the use of additional development languages. [The server currently has 4.4k stars and 1.4k forks on GitHub.](https://github.com/MariaDB/server) | -| Server Operating Systems | [FreeBSD, Linux, OS X, Solaris, and Windows](https://www.mysql.com/support/supportedplatforms/database.html)| [Linux and Windows](https://mariadb.com/docs/deploy/operating-systems/) | -| Compatibility | [MySQL and MariaDB have different views of JSON support.](https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility/#incompatibilities-between-currently-supported-mariadb-versions-and-mysql) MySQL uses the Internet Engineering Task Force (IETF) [RFC 7159](https://datatracker.ietf.org/doc/html/rfc7159) and [RFC 7396](https://datatracker.ietf.org/doc/html/rfc7396) standards for JSON support.| [Drop-in compatibility with MySQL up to version 5.5.](https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility/) Since then, new features make MariaDB increasingly incompatible with MySQL, so it’s important to verify compatibility before using MariaDB in an existing application. | -| Linux Distributions that include as part of Distribution | Unknown | Some of the most popular Linux distributions [include MariaDB by default](https://mariadb.com/kb/en/distributions-which-include-mariadb/): CentOS, Debian, Fedora, OpenSUSE, and Red Hat Enterprise. | -| Companies Using | [At least 5,878](https://stackshare.io/mysql) companies (not including development firms) currently use MySQL, including Uber, Airbnb, Shopify, Pinterest, Netflix, Amazon, Udemy, and Twitter. | The number of verified companies using MariaDB varies, but [featured customers](https://mariadb.com/resources/customer-stories/) include: Samsung, Virgin Media, Red Hat, Nokia, Select Quote, Tock, Walgreens, Pixid, Development Bank of Singapore (DBS), and Whitebox. | - - -## Download and Install MySQL Server - -The steps in this section detail installing MySQL on [CentOS Stream 8](https://linuxhint.com/what_is_centos_stream/). The process also works well with the older, and now deprecated, CentOS 8. The goal is to connect MySQL to CentOS 8, no matter what form it takes. Ultimately, you install MySQL server on Linux using the following steps: - -1. Update the system using the command below: - - ```command - sudo dnf update - ``` - - A message displays at the end with the number of packages that need to be upgraded. - - - You may see some messages during the update process, such as whether the upgrade requires additional disk space. If additional disk space is required, type `Y` and press **Enter** to continue. - - - A progress indicator is shown for each upgrade to keep the user apprised of how the process is going. - - - In case there is a kernel upgrade, reboot your system to reflect the changes after the upgrade. - - - Restart the services with outdated libraries using the GUI screens provided. - -1. Perform required package upgrades to keep the system up to date. - - ```command - sudo dnf upgrade - ``` - - During the update process, you may be asked questions such as if you want to use additional disk space. Also, a progress indicator is shown anytime the system performs an upgrade. Mostly, you see a message telling you that the upgrades completed at the same time as the updates, so there is nothing to upgrade. - -1. MySQL 8 is conveniently provided as part of the default repositories for CentOS Stream 8. You can directly run the command for MySQL set up as below. - - ```command - sudo dnf install @mysql - ``` - - As with the update process, you are asked questions like whether you want to use additional disk space and also see the progress indicator. - -1. Start the MySQL server using the command below: - - ```command - sudo systemctl start mysqld.service - ``` - -1. Verify the status of MySQL 8.0. - - ```command - sudo systemctl status mysqld - ``` - - You see output similar to that shown below: - - ```output - mysqld.service - MySQL 8.0 database server - Loaded: loaded (/usr/lib/systemd/system/mysqld.service; disabled; vendor preset: disabled) - Active: active (running) since Fri 2022-07-29 15:53:50 UTC; 52s ago - Process: 29625 ExecStartPost=/usr/libexec/mysql-check-upgrade (code=exited, status=0/SUCCESS) - Process: 29501 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mysqld.service (code=exited, status=0/SUCCESS) - Process: 29477 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS) - Main PID: 29581 (mysqld) - Status: "Server is operational" - Tasks: 38 (limit: 4921) - Memory: 456.1M - CGroup: /system.slice/mysqld.service - └─29581 /usr/libexec/mysqld --basedir=/usr - - Jul 29 15:53:37 172-105-17-53.ip.linodeusercontent.com systemd[1]: Starting MySQL 8.0 database server... - Jul 29 15:53:37 172-105-17-53.ip.linodeusercontent.com mysql-prepare-db-dir[29501]: Initializing MySQL database - Jul 29 15:53:50 172-105-17-53.ip.linodeusercontent.com systemd[1]: Started MySQL 8.0 database server. - ``` - -1. Allow MySQL 8 to automatically start every time you reboot your server using the command below: - - ```command - sudo systemctl enable mysqld - ``` - - Your MySQL server is now ready to use. - -## Configure MySQL Using MySQL Installation Script - -For a safer MySQL installation, use the `mysql_secure_installation` script to create a secure environment. The following steps show you how: - -1. Run `mysql_secure_installation` script. You may be asked to provide the `root` password. - - ```command - sudo mysql_secure_installation - ``` - - A welcome message is displayed on the screen and then the option to install `VALIDATE PASSWORD COMPONENT`. This component verifies that users are relying on strong passwords to log into MySQL, so it’s an important addition to your security suite. - -1. Type `Y` and press **Enter** to install the component. If you choose to install `VALIDATE PASSWORD COMPONENT`, go through the following series of sub-steps to install the same: - - - Firstly you need to set the password validation policy. Here, `0` is for low, `1` for medium, and `2` for strong. Setting a strong password option is the best option because it requires a password length of at least eight characters and the use of numeric, mixed case, special characters, and words that don’t appear in the dictionary. Enter a numeric value in a valid range of `0` and `3`, then press `Enter`. - - - Enter the `root` user password. Type a password that matches the validation policy you set and press **Enter**. You are shown the strength of the password you entered, with 100 being quite strong. In case you do not wish to change the password, press **Enter** to signify no changes in the MySQL `root` user’s password. - - - Type `Y` and press **Enter** if you’re happy with the password. - - - Now, you are prompted if you want to remove anonymous users. Type `Y` and press **Enter** unless you want to allow users without authentication to access your server. - - - Further, you are prompted if you want to disallow remote logins. Remote login access is required to access the MySQL server from outside, so press **Enter** to signify no. - - - Next in the script, it prompts the user to remove the test database. The test database can be helpful during the experimental phase of MySQL. If you wish to remove the test database type `Y` and press **Enter**. Press **Enter** if you want to keep the test database. - - - The next prompt is important because you need to reload the privilege tables for the changes to become permanent and have immediate effect. Type `Y` and press **Enter**. MySQL is now configured. - -## Start, Stop, and Restart MySQL on CentOS Stream 8 - -As you work with MySQL on CentOS Stream 8, ensure that the server is in the expected state. Check the status of the MySQL server using the command below: - -```command -sudo systemctl status mysqld -``` - -The output shows MySQL as started and enabled. The following is a list of helpful commands to start, stop, and restart MySQL as needed: - -- **Start MySQL server**: - -```command -sudo systemctl start mysqld.service -sudo systemctl enable mysqld -``` - -- **Stop MySQL server**: - -```command -sudo systemctl stop mysqld.service -sudo systemctl disable mysqld -``` - -- **Restart MySQl server**: - -```command -sudo systemctl restart mysqld.service -``` - -## Test the Installation - -To run a quick test of your MySQL installation, run the below command to verify the service runtime status. - -```command -sudo systemctl status mysqld -``` - -Then, perform the following checks: - -1. **Check MySQL version**: Use the command below to check the version of MySQL installed. You are asked for the password that you created earlier. If the entered password is successful, you see information about your version of MySQL. - - ```command - mysqladmin -u root -p version - ``` - -1. **Access MySQL server**: You can access the MySQL server with the following sub-steps: - - - Login to MySQL as the `root` user. Provide the root user password when prompted. - - ```command - mysql -u root -p - ``` - - - Create a database named `Test` in MySQL server. - - ```command - CREATE DATABASE Test; - ``` - - {{< note >}} -Don’t forget the semicolon at the end of the command. - {{< /note >}} - - MySQL gives the following output: - - ```output - Query OK, 1 rows affected (0.00 sec) - ``` - - - To check the list of current databases, which includes the ``Test`` database, on MySQL, run the command below: - - ```command - SHOW DATABASES; - ``` - - - To remove a database from the server, run the command below: - - ```command - DROP DATABASE Test; - ``` - - - Again, check the list of databases present on the MySQL server. Now you have the latest list of available databases. You see `Test` database deleted in the previous step is missing from the list. - - ```command - SHOW DATABASES; - ``` - - - Exit MySQL and return to the terminal prompt. - - ```command - EXIT - ``` - -## Conclusion - -One of the biggest takeaways is that both MySQL and MariaDB provide enterprise-level database functionality. Each has its specialization. Installing either product is relatively easy using the Package Manager. When installing MySQL, take additional steps when working with the [MySQL installation script](/cloud/guides/how-to-install-mysql-on-centos8#configure-mysql-using-mysql-installation-script). Remote access to MySQL setup requires that you configure MySQL to allow remote login and then set up UFW as well. diff --git a/docs/guides/databases/postgresql/how-to-install-postgresql-relational-databases-on-centos-7/index.md b/docs/guides/databases/postgresql/how-to-install-postgresql-relational-databases-on-centos-7/index.md deleted file mode 100644 index 3b1fed081d2..00000000000 --- a/docs/guides/databases/postgresql/how-to-install-postgresql-relational-databases-on-centos-7/index.md +++ /dev/null @@ -1,478 +0,0 @@ ---- -slug: how-to-install-postgresql-relational-databases-on-centos-7 -title: 'How to Install PostgreSQL Relational Databases on CentOS 7' -description: 'The PostgreSQL relational database is a powerful open source database platform. Learn how to install it on CentOS 7 in this simple tutorial.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2016-12-12 -keywords: ["postgresql", "CentOS 7", "open source database", "relational database"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[PostgreSQL Online Documentation](http://www.postgresql.org/docs/)' - - '[psql manual page](http://www.rootr.net/man/man/psql/1)' -relations: - platform: - key: use-postrgesql-database - keywords: - - distribution: CentOS 7 -aliases: [] -tags: ["database","postgresql","centos"] ---- - -The [PostgreSQL](http://www.postgresql.org/) relational database system is a powerful, scalable, and standards-compliant open-source database platform. This guide will help you install and configure PostgreSQL on your CentOS 7 Linode. - -![Use PostgreSQL Relational Databases on CentOS 7](use-postgresql-on-centos-7-title.png "Use PostgreSQL Relational Databases on CentOS 7") - -## Before You Begin - -1. Familiarize yourself with our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) and complete the steps for setting your Linode's hostname and timezone. - -2. Complete the sections of our [Securing Your Server guide](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) to create a standard user account, harden SSH access and remove unnecessary network services. - -3. Update your system: - - sudo yum update - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, visit the [Users and Groups guide](/cloud/guides/linux-users-and-groups) for more information. -{{< /note >}} - -## Install PostgreSQL - -In this section, we'll cover two different methods for installing PostgreSQL. If you do not need the latest version, we recommend using the first method to install PostgreSQL from the CentOS repositories. - -Unless otherwise noted, the instructions in subsequent sections of this guide will be compatible with versions installed by either method. - -### Install From the CentOS Repositories - -1. As of this writing, the CentOS 7 repositories ship with PostgreSQL version `9.2.15`. To install from the CentOS repositories, simply run: - - sudo yum install postgresql-server postgresql-contrib - -2. Initialize your Postgres database and start PostgreSQL: - - sudo postgresql-setup initdb - sudo systemctl start postgresql - -3. **Optional**: Configure PostgreSQL to start on boot: - - sudo systemctl enable postgresql - -### Install From the Postgres Repositories - -Alternatively, you can install the latest version from the Postgres repositories. As of this publication, PostgreSQL `9.6.3` is the most recent version available for CentOS 7, but these steps can be applied to any RPM-based installation. - -{{< note >}} -When Postgres is installed using this method, the version number is included in its configuration directories. For example, `/var/lib/pgsql` becomes `/var/lib/pgsql/9.6`. This is also the case with systemd units; `systemctl status postgresql` becomes `systemctl status postgresql-9.6`. -{{< /note >}} - -1. Select the version you wish to install from the [Postgres Yum repositories](https://yum.postgresql.org/repopackages.php). Locate the CentOS 7 link for your chosen version and download it to your Linode: - - wget https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-centos96-9.6-3.noarch.rpm - -2. Install the RPM, as well as the [EPEL](https://fedoraproject.org/wiki/EPEL) repositories, which will be used to satisfy dependencies: - - sudo yum install pgdg-centos96-9.6-3.noarch.rpm epel-release - -3. Update Yum to apply your changes and install PostgreSQL. When installing Postgres manually, you will have to specify the version: - - sudo yum update - sudo yum install postgresql96-server postgresql96-contrib - -4. Initialize your database and start PostgreSQL: - - sudo /usr/pgsql-9.6/bin/postgresql96-setup initdb - sudo systemctl start postgresql-9.6 - -5. **Optional**: Configure PostgreSQL to start on boot: - - sudo systemctl enable postgresql-9.6 - -## Configure PostgreSQL - -### Secure the Postgres Users - -By default, PostgreSQL will create a Linux user named `postgres` to access the database software. - -{{< note type="alert" >}} -The `postgres` user should not be used for other purposes (e.g., connecting to other networks). Doing so presents a serious risk to the security of your databases. -{{< /note >}} - -1. Change the `postgres` user's Linux password: - - sudo passwd postgres - -2. Issue the following commands to set a password for the `postgres` database user. Be sure to replace `newpassword` with a strong password and keep it in a secure place. - - su - postgres - psql -d template1 -c "ALTER USER postgres WITH PASSWORD 'newpassword';" - - Note that this user is distinct from the `postgres` Linux user. The Linux user is used to access the database, and the PostgreSQL user is used to perform administrative tasks on the databases. - - The password set in this step will be used to connect to the database via the network. Peer authentication will be used by default for local connections. See the [Secure Local PostgreSQL Access section](/cloud/guides/how-to-install-postgresql-relational-databases-on-centos-7#secure-local-access) for information about changing this setting. - -### Access the PostgreSQL Shell - -The PostgreSQL client shell allows you to issue SQL commands to administer your databases. As the `postgres` Linux user, log in by running: - - psql postgres - -This will log you in as the `postgres` database user. You'll see a prompt similar to this: - - psql (9.2.15) - Type "help" for help. - - postgres=# - -In the last line, `postgres=#` indicates the name of the current database. To see a list of available commands, type `\h`. You may find more information on a specific command by adding it after `\h`. Once you’ve finished using the shell, you can exit with `\q`. - -## Work with Databases - -This section will cover how to create, delete and access databases. - -### Create a Database - -You can create databases with the `createdb` command. Create a sample database called `mytestdb` by running this command as the `postgres` Linux user: - - createdb mytestdb - -It's also possible to assign ownership of the database to a specific Postgres user/role. For example, you could assign ownership to the `examplerole` role by running: - - createdb mytestdb -O examplerole - -The `createdb` command has several additional options, which can be found in the [PostgreSQL documentation](https://www.postgresql.org/docs/9.2/static/app-createdb.html). - -### Connect to a Database - -You can use the `psql` command to connect to a specific database. - -1. Connect to the test database: - - psql mytestdb - -2. You will see the following output: - - psql (9.2.15) - Type "help" for help. - - mytestdb=# - - By default, you will connect to a database as your [peer-authenticated](#peer-authentication) user. However, if you've enabled [local password access](#secure-local-access), it's also possible to specify which user you wish to connect as: - - psql mytestdb -U examplerole - - You'll be prompted to enter the password for the `examplerole` database user before you access the shell. - -### List Databases - -From the [Postgres shell](#access-the-postgresql-shell), you can list all of your databases with the `\l` or `\list` command. You will receive output similar to this: - - postgres=# \l - List of databases - Name | Owner | Encoding | Collate | Ctype | Access privileges - -----------+----------+----------+-------------+-------------+----------------------- - mytestdb | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | - postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | - template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + - | | | | | postgres=CTc/postgres - template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + - | | | | | postgres=CTc/postgres - (4 rows) - -You may also show the current database and user by entering `\c` from the Postgres shell. Additional info, like socket and port, will be included if you use `\conninfo`: - - You are connected to database "mytestdb" as user "postgres" via socket in "/var/run/postgresql" at port "5432". - -### Delete a Database - -You can delete, or *drop*, databases with the `dropdb` command. For example, to delete the `mytestdb` database created previously, issue this command as the `postgres` Linux user: - - dropdb mytestdb - -{{< note type="alert" >}} -Deleted databases cannot be recovered. -{{< /note >}} - -## Work With Tables - -PostgreSQL databases use tables to store and organize information within a database. In this section, you'll find practical examples for adding, removing and manipulating tables. Unless otherwise noted, the commands in this section should be issued from the Postgres shell once you've [connected to your database](#connect-to-a-database). - -### Create Tables - -This section contains examples that create a test database with an employee's first and last name, assigning each a unique key. When creating your own tables, you may specify as many parameters (columns) as you need and name them appropriately. - -1. Create a table called "employees" in your test database: - - CREATE TABLE employees (employee_id int, first_name varchar, last_name varchar); - -2. Insert a record into the table: - - INSERT INTO employees VALUES (1, 'John', 'Doe'); - -### View the Content of a Table - -To view the contents of the "employees" table: - - SELECT * FROM employees; - -This produces the following output: - - employee_id | first_name | last_name - -------------+------------+----------- - 1 | John | Doe - (1 row) - -### List Tables in a Database - -You can list all tables in the current database with the `\dt` command: - - mytestdb-# \dt - List of relations - Schema | Name | Type | Owner - --------+-----------+-------+---------- - public | employees | table | postgres - -### Delete Tables - -Delete tables with `DROP TABLE`. To delete the `employees` table: - - DROP TABLE employees; - -`DROP TABLE` accepts multiple comma-separated table names as arguments. For example, if you had two separate tables called `employees1` and `employees2`, you could delete them both by running: - - DROP TABLE employees1, employees2; - -### Add Columns - -Tables can be altered to add definitions, data types and columns. In this example you'll add a new `start_date` column that uses the [date](https://www.postgresql.org/docs/9.2/static/datatype-datetime.html) data type. - -1. Add the `start_date` column to the `employees` table: - - ALTER TABLE employees ADD start_date date; - -2. Verify your change: - - SELECT * FROM employees; - - You'll see that the new column has been created, but it does not contain any data: - - employee_id | first_name | last_name | start_date - -------------+------------+-----------+------------ - 1 | John | Doe | - (1 row) - - In this example you've used the `date` data type, but PostgreSQL tables support several different types of data. See the [PostgreSQL Documentation](https://www.postgresql.org/docs/9.2/static/datatype.html) for a full explanation of supported data types. - -### Add and Update Rows - -In this section, you'll use `UPDATE` to enter a value into the existing row you've created. Then, you'll create an entirely new row with `INSERT`. - -1. Update the `start_date` field for the user with the value `1` in the `employee_id` column: - - UPDATE employees SET start_date = '2016-09-28' WHERE employee_id = '1'; - -2. Create a new row in the `employees` table: - - INSERT INTO employees VALUES (2, 'Jane', 'Smith', '2015-03-09'); - -3. Verify your changes: - - SELECT * FROM employees; - - You'll see that the start date of `2016-09-28` has been added to the first row, and that a new row has been created for "Jane Smith": - - employee_id | first_name | last_name | start_date - -------------+------------+-----------+------------ - 1 | John | Doe | 2016-09-28 - 2 | Jane | Smith | 2015-03-09 - (2 rows) - -### Remove Columns and Rows - -In this section, you'll remove a column from your table and then remove the second row. - -1. Use `ALTER TABLE` to remove the `start_date` column you made previously: - - ALTER TABLE employees DROP start_date; - -2. Now use `DELETE` to remove the second row of your `employees` table. The following command will remove the row with a value of `2` in the `employee_id` column: - - DELETE FROM employees WHERE employee_id = '2'; - -3. Confirm your changes: - - SELECT * FROM employees; - - Your table now consists of a single row, with the `start_date` column removed: - - employee_id | first_name | last_name - -------------+------------+----------- - 1 | John | Doe - (1 row) - -### Query a Table - -You can use queries to pull specific information from your database. This command will query your `employees` table to only return values for the `employee_id` and `last_name` columns: - - SELECT last_name,employee_id FROM employees; - -You'll receive an output similar to this: - - last_name | employee_id - -----------+------------- - Doe | 1 - (1 row) - -PostgreSQL supports many querying options. See the [PostgreSQL Documentation](https://www.postgresql.org/docs/9.2/static/sql-select.html) for more information. - -## Work With Roles - -PostgreSQL grants database access via **roles**, which are used to specify privileges. Roles can be understood as having a similar function to Linux "users." In addition, roles may also be created as a set of other roles, similar to a Linux "group." PostgreSQL roles apply globally, so you will not need to create the same role twice if you'd like to grant it access to more than one database on the same server. - -### Create Roles - -New user roles are added with the `createuser` command. To create a new user called `examplerole`, issue this command as the `postgres` Linux user: - - createuser examplerole --pwprompt - -You will be prompted to create a password for the new user. - -### Give a Role Access to a Database - -In this example, you'll give the newly created `examplerole` user access to your database. - -1. Connect to the database: - - psql mytestdb - - You'll be connected as the `postgres` database user by default. - -2. From the PostgreSQL shell, enter the following to grant all privileges on the table `employees` to the user `examplerole`: - - GRANT ALL ON employees TO examplerole; - -3. Exit the database with `\q`. - -### List All Roles - -You can list all roles from the [Postgres Shell](#access-the-postgresql-shell) by running `\du`. You'll see an output similar to this: - - postgres=# \du - List of roles - Role name | Attributes | Member of - -------------+------------------------------------------------+----------- - examplerole | | {} - postgres | Superuser, Create role, Create DB, Replication | {} - -### Group Roles - -For ease of administration, it's possible to add multiple user roles to a single group, so that their privileges can be managed as a whole. In this section you'll create a new group and add the `examplerole` user to it. These commands should be run as the `postgres` Linux user. - -1. Use the `createuser` command to create a new group role. The `--no-login` option is specified because groups do not need login capability. - - createuser examplegroup --no-login - -2. Log into the Postgres shell and add `examplerole` to the new group: - - psql postgres - GRANT examplegroup TO examplerole; - -3. From the Postgres shell, verify your changes with `\du`. You'll see that the `examplerole` user is now listed as a member of the `examplegroup` group: - - postgres=# \du - List of roles - Role name | Attributes | Member of - --------------+------------------------------------------------+---------------- - examplegroup | Cannot login | {} - examplerole | | {examplegroup} - group | | {} - postgres | Superuser, Create role, Create DB, Replication | {} - - The `createuser` command has several other options. See the [PostgreSQL documentation](https://www.postgresql.org/docs/9.2/static/app-createuser.html) for more details. - -4. When you've finished applying your changes, exit the Postgres shell with `\q`. - -### Alter Roles - -While specific settings and privileges can be applied to a role when it's created, you can also modify a role's properties later on. In this example, we'll modify the `examplerole` user so that it can create new databases. The commands in this section should be run as the `postgres` Linux user. - -1. Log in as the `postgres` database user: - - psql postgres - -2. From the Postgres shell, add the `CREATEDB` parameter to the `examplerole` user: - - ALTER ROLE examplerole CREATEDB; - - A number of permissions can be applied when creating or altering a role. See the [PostgreSQL Documentation](https://www.postgresql.org/docs/9.2/static/sql-createrole.html) for more details. - -3. Use `\du` to confirm your changes. You'll see that the "Create DB" attribute is listed next to the `examplerole` user: - - postgres=# \du - List of roles - Role name | Attributes | Member of - -------------+------------------------------------------------+----------- - examplerole | Create DB | {} - group | | {} - postgres | Superuser, Create role, Create DB, Replication | {} - -4. Once you've finished, exit the Postgres shell with `\q`. - -### Delete Roles - -The `dropuser` command is used to delete PostgreSQL roles. To delete the `examplerole` user, issue this command as the `postgres` Linux user: - - dropuser examplerole - -### Peer Authentication - -PostgreSQL uses **peer authentication** by default. This means that database connections will be granted to local system users if their Linux username matches the name of their PostgreSQL role. To make use of peer authentication effectively, you would need to create both a Linux user and a corresponding PostgreSQL role. For the `examplerole` role you just created, you can use peer authentication by creating an `examplerole` local system user. This command must be run as a user with `sudo` access: - - sudo adduser examplerole && passwd examplerole - -Note that you will be prompted to create a password for the new `examplerole` Linux user. Alternatively, you can follow our steps to [secure local access](#secure-local-access). - -## Secure PostgreSQL - -### Secure Local Access - -While PostgreSQL's default peer authentication is useful in cases where a particular system user will be running a local program (e.g., scripts, CGI/FastCGI processes owned by separate users, etc.), you may wish to require passwords for greater security. - -Commands in this section should be run as the `postgres` Linux user unless otherwise specified. - -1. Edit the `/var/lib/pgsql/data/pg_hba.conf` file, under the `# "local" is for Unix domain socket connections only` header: - - {{< file "/var/lib/pgsql/data/pg_hba.conf" >}} -# "local" is for Unix domain socket connections only -local all all peer - -{{< /file >}} - - - Replace `peer` with `md5` on this line to activate password authentication using an MD5 hash. - - {{< note respectIndent=false >}} -If you installed PostgreSQL from the [Postgres repositories](#install-from-the-postgres-repositories), you will need to specify your version number in this file path, for example: `/var/lib/pgsql/9.6/data/pg_hba.conf`. -{{< /note >}} - -2. To enable these changes, you need to restart PostgreSQL. However, you did not grant the `postgres` user sudo privileges for security reasons. Return to the normal user shell: - - exit - -3. Restart PostgreSQL and switch back to the `postgres` user: - - sudo systemctl restart postgresql - su - postgres - -4. As `postgres`, connect to the test database as the `examplerole` PostgreSQL user: - - psql mytestdb -U examplerole - - You will be prompted to enter the password for the `examplerole` user and then given `psql` shell access to the database. When using a database, you may check access privileges for each of its tables with the `\z` command. - -### Secure Remote Access - -PostgreSQL listens for connections on `localhost` by default, and it is not advised to reconfigure it to listen on public IP addresses. If you wish to make PostgreSQL externally accessible, it's recommended that you follow the Postgres documentation for [using SSL](https://www.postgresql.org/docs/9.2/static/ssl-tcp.html) to secure your remote connections. Alternatively, you could connect to PostgreSQL over an [SSH tunnel](https://www.postgresql.org/docs/9.2/static/ssh-tunnels.html). To access your databases remotely using a graphical tool, please follow one of these guides: - -- [Securely Manage Remote PostgreSQL Servers with pgAdmin on Windows](/cloud/guides/how-to-access-postgresql-database-remotely-using-pgadmin-on-windows) -- [Securely Manage Remote PostgreSQL Servers with pgAdmin on Mac OS X](/cloud/guides/securely-manage-remote-postgresql-servers-with-pgadmin-on-macos-x) diff --git a/docs/guides/databases/redis/lua-scripting-for-redis/index.md b/docs/guides/databases/redis/lua-scripting-for-redis/index.md deleted file mode 100644 index 28f2f3e38cd..00000000000 --- a/docs/guides/databases/redis/lua-scripting-for-redis/index.md +++ /dev/null @@ -1,193 +0,0 @@ ---- -slug: lua-scripting-for-redis -title: "Writing Lua Scripts for a Redis Server" -description: "In this tutorial learn how to start writing and using Lua scripts to improve your Redis database experience." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2023-03-14 -keywords: ['redis lua scripts','redis lua script tutorial','redis lua examples'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Redis: Scripting with Lua](https://redis.io/docs/manual/programmability/eval-intro/)' -- '[Redis: EVAL](https://redis.io/commands/eval/)' -- '[freeCodeCamp: A Quick Guide to Redis Lua Scripting](https://www.freecodecamp.org/news/a-quick-guide-to-redis-lua-scripting/)' ---- - -Redis is an open source NoSQL database that offers low-latency in-memory storage of data structures. Web application developers find Redis a powerful ally for application caching, messaging, and many other operations. - -Redis uses Lua for scripting, letting you consolidate more complicated tasks in an efficient way. Script logic gets executed on the database server, allowing them to be reused, and often increasing performance. - -In this tutorial learn what Redis' Lua scripting has to offer and how you can start using it on your Redis database. The tutorial covers the pros and cons of Redis scripting, creating and deploying scripts, and managing cached scripts. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. Follow the instructions in our [How to Install a Redis Server](/cloud/guides/how-to-install-a-redis-server-on-ubuntu-or-debian8) guide to install a Redis server and command line interface (CLI). Be sure to use the drop-down menu at the top of the page to select your Linux distribution and view the appropriate steps. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Why Use Lua Scripting on a Redis Server? - -Scripting allows you to execute complex tasks within Redis. A Redis script allows you to use programming tools like control structures, while having access to almost all Redis commands. - -So why would you use a Redis script over executing commands directly, or from your application itself? Because with scripts: - -- Tasks are executed directly on the database server, often increasing their performance substantially. - -- Logic lives directly on the database server, which can be useful for logic used across multiple applications. - -- Steps get executed atomically, meaning that the server prevents other actions while a script is still running. - -Lua is the language for Redis scripting. The Lua language offers simplicity and conciseness, making it an effective language for scripting tasks. - -However, scripts are not suited to all cases. Because the Redis server blocks other operations while a script is running, lengthy scripts may actually result in negative performance impacts. - -Scripting also executes logic on the database. Doing so can lead to bad architectural and design practices when developers rely on it to implement application logic rather than data-level logic. - -## How to Write Lua Scripts for a Redis Server - -Basic Redis scripts can be executed using the `EVAL` command. The command executes scripts directly from the Redis CLI: - -```command -EVAL "return 'Hello, world!'" 0 -``` - -```output -"Hello, world!" -``` - -Redis' Lua implementation has access to the `redis.call` method, allowing you to execute most Redis commands from the Lua script. In this example, the Lua script executes the `SET` command: - -```command -EVAL "redis.call('SET', KEYS[1], ARGV[1])" 1 example_key "Example Value" -``` - -The `EVAL` command has a minimum of two arguments, and more if the second argument is greater than zero. - -- The first argument is the *script* itself. The Lua code for the script is wrapped in quotation marks. - -- The second argument is the *number of keys*. This is `1` in the example above, indicating that one key-value pair is used in the script. That key is identified with `KEY[1]` in the script itself. Its corresponding value is indicated with `ARGV[1]`. - -- The third argument contains *key names*. This is `example_key` in the example above. When the number of keys is greater than one, you need to provide a number of keys here equal to the number of keys argument. - -- The fourth argument contains *values*. Above, this is `"Example Value"`. As with the key names argument, you need to provide an array of values equal to the number of keys. - -A similar script with multiple keys and value could be: - -```command -EVAL "redis.call('SET', KEYS[1], ARGV[1]); redis.call('SET', KEYS[2], ARGV[2])" 2 example_key_1 example_key_2 "Example Value 1" "Example Value 2" -``` - -You do not always need to provide a value. For example, when using a read command like `GET`: - -```command -EVAL "return redis.call('GET', KEYS[1])" 1 example_key_1 -``` - -```output -"Example Value 1" -``` - -More advanced usage leverages the capabilities of the Lua language. Here is a simple example to give you an idea: - -```command -EVAL "local key_name = 'example_key'; for iterated_value=0,4 do redis.call('hmset', KEYS[1], key_name .. tostring(iterated_value), iterated_value) end; return redis.call('hgetall', KEYS[1])" 1 example_hash -``` - -```output - 1) "example_key0" - 2) "0" - 3) "example_key1" - 4) "1" - 5) "example_key2" - 6) "2" - 7) "example_key3" - 8) "3" - 9) "example_key4" -10) "4" -``` - -## How to Store Lua Scripts on a Redis Server - -Redis scripts can be pre-loaded into memory for later execution. This feature is useful especially for scripts that need to be executed numerous times. This feature still allows you to provide different parameters for each script execution. - -Using the `SCRIPT LOAD` command stores a given script in Redis' cache. Notice that you do not provide the script with parameters at this time. Instead, you provide the parameters whenever you call the script, as you can see further on: - -```command -SCRIPT LOAD "redis.call('set', KEYS[1], ARGV[1]); return redis.call('get', KEYS[1])" -``` - -```output -"5afd00504d9d21a8fc37cd1b4400872d2e69296a" -``` - -Redis provides you with an SHA1 identifier for the new script. You can use that identifier with the `EVALSHA` command to execute the script. The command takes the SHA1 identifier and the arguments for the script just as the `EVAL` command would: - -```command -EVALSHA 5afd00504d9d21a8fc37cd1b4400872d2e69296a 1 example_key "Another example value" -``` - -```output -"Another example value" -``` - -The script cache is volatile and not persisted. Thus, after some time or certain server events, the `EVALSHA` command may return an error, indicating that the identified script has been purged from the cache: - -```output -(error) NOSCRIPT No matching script. Please use EVAL. -``` - -## How to Use the Script Command to Manage Lua Scripts - -Redis provides several `SCRIPT` commands that give you a degree of control over scripts stored in the cache, such as the `SCRIPT LOAD` command in the previous section. - -The following list shows the other `SCRIPT` commands used to manage scripts in cache. - -- `SCRIPT EXISTS` lets you verify whether a script with a given SHA1 identifier exists. Above, you can see that the `EVALSHA` command inherently indicates whether a script exists or not. But you can achieve cleaner and clearer code in some cases by using the `SCRIPT EXISTS` command to explicitly check for a script: - - ```command - SCRIPT EXISTS "5afd00504d9d21a8fc37cd1b4400872d2e69296a" - ``` - - ```output - 1) (integer) 1 - ``` - - This response indicates that the script exists. A `0` response would indicate that the script does not exist in cache. Multiple SHA1 identifiers can be provided to the command, in which case the command returns multiple values of `1` and/or `0`, corresponding to each identifier. - -- `SCRIPT DEBUG` can be used to enable/disable debugging mode for Lua scripts. When debugging is enabled, `EVAL` commands use the Lua debugger built into Redis. - - The debugging mode comes with two options. It can be run *asynchronously* using `SCRIPT DEBUG YES`. This has debugging operate on a separate session where it does not block Redis operations. With this option, changes are rolled back when the script finishes. Alternatively, debugging mode can be run *synchronously* using `SCRIPT DEBUG SYNC`. The script in this case blocks other Redis operations and saves changes just like scripts in normal mode. - - Most often, asynchronous debugging is preferable, as it lets you run scripts without significant consequence. - - Redis recommends script debugging not be used in production environments. Using a testing or development server instead can prevent unforeseen consequences and performance impacts. - - ```command - SCRIPT DEBUG YES - EVAL "redis.call('set', KEYS[1], ARGV[1])" 1 another_example_key "A value" - SCRIPT DEBUG NO - ``` - -- `SCRIPT KILL` provides the only way to interrupt a running script, other than restarting the server. The command is used to stop scripts that have run over the maximum execution time. However, this command only works for scripts that have not yet modified any data: - - ```command - SCRIPT KILL - ``` - -- `SCRIPT FLUSH` clears Redis' script cache. This can be especially useful when working in a test environment, allowing you to clear out the cache for a new set of scripts to test: - - ```command - SCRIPT FLUSH - ``` - -## Conclusion - -You are now prepared to step into the realm of Lua scripting on your Redis server. This tutorial covers everything you need to get started, from writing scripts to deploying and managing them. - -Want to continue learning about Redis to get the most effective use out of your server? We have plenty of [guides on using Redis](/cloud/guides/databases/redis) that can help you navigate Redis data types, configurations, and more. \ No newline at end of file diff --git a/docs/guides/databases/redis/redis-client-side-caching/index.md b/docs/guides/databases/redis/redis-client-side-caching/index.md deleted file mode 100644 index b5218546a96..00000000000 --- a/docs/guides/databases/redis/redis-client-side-caching/index.md +++ /dev/null @@ -1,297 +0,0 @@ ---- -slug: redis-client-side-caching -title: "Use Redis for Client-Side Caching" -title_meta: "How to Use Redis for Client-Side Caching" -description: "Modern web applications rely on client-side caching to enhance its performance. This guide shows you how to use Redis for server-assisted client-side caching." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2022-04-08 -keywords: ['redis caching tutorial','redis server assisted client-side caching','how redis caching works'] -tags: ['redis'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Redis: Redis server-assisted client side caching](https://redis.io/topics/client-side-caching)' -- '[MDN Web Docs: Client-side Storage](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Client-side_web_APIs/Client-side_storage)' ---- - -Redis is an open-source in-memory database with a reputation for working exceptionally well for caching web application data. Redis's fast transactions and low latency help to increase application performance while its server-assisted caching feature makes client-side caching more efficient. - -This tutorial explains the concepts behind Redis's server-assisted client-side caching, including how client-side caching and server assistance work. The guide then breaks down how to set up your Redis server to get the most out of client-side caching in your web applications. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's hostname and timezone. - -1. This guide uses `sudo` wherever possible. Complete the sections of our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to create a standard user account, harden SSH access, and remove unnecessary network services. - -1. Update your system. - - - On **Debian** and **Ubuntu**, use the following command: - - sudo apt update && sudo apt upgrade - - - On **AlmaLinux**, **CentOS** (8 or later), or **Fedora**, use the following command: - - sudo dnf upgrade - -1. Follow the instructions in our [How to Install and Configure Redis](/cloud/guides/install-redis-ubuntu) guide to installing a Redis server and command-line interface (CLI). Be sure to use the drop-down menu at the top of that page to select your Linux distribution and get the appropriate steps. - -{{< note >}} -The steps in this guide is written for non-root users. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Linux Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## What Is Client-Side Caching? - -To improve performance, many modern web applications utilize the browser's ability to store some data locally. Doing so is called *client-side caching*, and serves the purpose, mainly, of improving performance. - -Rather than making call to the server every time some data is needed, an application can first check the local cache. If the data is there, the application does not need to call the server for it. If the data is not there yet, the application can fetch the data and store it in the cache for later. - -Browser cookies are an example of client-side storage. They allow an application to store data like user login and site preferences. But modern browsers also support other means of storage, making client-side caching more versatile and capable. - -### Benefits of Client-Side Caching - -The main perk to client-side caching is performance. Web applications that use client-side caching avoid repetitive calls to the server, improving the speed of data retrieval and reducing their overall network loads. - -However, the client-side caches suffer when it comes to keeping cached data up to date. The use of a local cache introduces the possibility of having stored data become outdated and out of sync with the data on the server. - -This may not be an issue for some use cases. For instance, you may have reasonable certainty about how often data changes or how time-sensitive the data is. In such cases, you can mark the cached data with an invalidation timestamp, after which the application fetches fresh data from the server. - -But this limitation can be prohibitive for applications where data changes frequently and is more time-sensitive. In those cases, web applications often have to implement some other means of expiring local caches. - -## What Is Server-Assisted Client-Side Caching? - -To address the issue of updating or invalidating cached data, Redis implements a server-assisted model. - -In this model, the Redis server tracks the data used by each connected client. When some tracked data gets altered on the server, Redis sends out notifications to the relevant clients to invalidate the old data. - -Redis's model allows an application to get the performance benefits of caching while ensuring that the local data can be kept up to date. - -To elaborate, an application using Redis for server-assisted caching should use the following layers: - -- An *application layer* that fetches data either from the cache or, if the necessary data is not there, from the database server. Any data received from the database gets stored in the cache. - -- A *cache layer* that holds data fetched from the database server by the application. To prevent the data from becoming outdated, the application must remove or otherwise update cached data whenever that data changes. - -- A *database layer* that responds to requests for data from the application. Additionally, this layer provides invalidation notifications to the application when previously fetched data gets altered, helping the application to keep its cache up to date. - -![Diagram of a server-assisted client-side caching architecture](server-assisted-architecture.png) - -## How to Set Up Redis for Client-Side Caching - -Configuring server-assisted caching in Redis is done on a client-by-client basis. The next section walks you through the precise steps you need to get your Redis clients running with tracked caches. - -Further on, you can find some additional options for Redis's client tracking. This includes a broadcast mode, for tracking set of keys, and some parameters for more finely controlled cache tracking. - -The examples in the sections below assume a Redis database with some existing keys. You can use the following commands to create the keys used in these examples: - - SET cat_one:key_one "Anaphora" - SET cat_one:key_two "Bathos" - SET cat_one:key_three "Catharsis" - -### Configuring Redis Clients for Server-Assisted Caching - -The latest versions of Redis have two options for handling invalidation notifications: - -- Using a separate, dedicated client for monitoring notifications -- Receiving invalidation notifications directly on the tracking client via the *RESP3 protocol* - -Each option has its use cases, although the latter is only available on Redis version 6 onward. - -Below, you can see the steps used for setting up your clients with each of these options. - -#### Dedicated Client for Monitoring Notifications - -1. Create a Redis client by making an authenticated connection to the Redis server. See our guide on [How to Connect to Redis and Use The Redis Database](/cloud/guides/redis-getting-started) for instructions on doing so. - -1. Determine the client's ID using the following command. This ID is used in setting up client tracking in one of the subsequent steps, so keep note of it. This and subsequent examples use `15` for the notification client's ID. - - CLIENT ID - - {{< output >}} -(integer) 15 - {{< /output >}} - -1. Have the client subscribe to the invalidation message channel. Doing so sets the client into listening mode, meaning that it cannot issue commands but receives all messages on the subscribed channel. - - SUBSCIRBE __redis__:invalidate - - {{< output >}} -1) "subscribe" -2) "__redis__:invalidate" -3) (integer) 1 - {{< /output >}} - -1. Create another client, making another authenticated connection to the Redis server. - -1. Turn on client tracking on the new client. Use the `REDIRECT` option to have the client's invalidation messages sent to the first client's ID. - - CLIENT TRACKING on REDIRECT 15 - - The second client, and any subsequent clients that you set up in a similar way, are now being tracked for server-assisted caching. The server keeps note of any data that these tracked clients fetch. Then, the server sends invalidation messages to the listening client whenever any of that data changes. - -1. Verify that the second client is being tracker for server-assisted caching by issuing the following commands to the second client: - - GET cat_one:key_one - - {{< output >}} -"Anaphora" - {{< /output >}} - - SET cat_one:key_one "Anapest" - - {{< output >}} -OK - {{< /output >}} - - Now, on the first client, you should see a notification like the following: - - {{< output >}} -1) "message" -2) "__redis__:invalidate" -3) 1) "cat_one:key_one" - {{< /output >}} - -#### Using the RESP3 Protocol - -Redis 6 introduced the RESP3 protocol to provide more capabilities for Redis clients. - -Among these is the ability to receive messages from an active client. Previously (under the RESP2 protocol), a Redis client could only receive messages by subscribing to a channel. This left the client in listening mode, unable to issue commands. - -RESP3 makes it possible for a client to be simultaneously open for issuing commands and set to listen for incoming messages, like invalidation notices. - -By default, Redis clients start in the RESP2 protocol. Below, you can see steps for enabling RESP3 and using it for receiving invalidation messages on a Redis client. - -{{< note >}} -Redis clients must be version 6 or later to use RESP3. Check your Redis version before proceeding: - - redis-cli --version - -For Redis versions less than 6, see the previous section for [creating a dedicated client for invalidation messages](#dedicated-client-for-monitoring-notifications). -{{< /note >}} - -1. Create a Redis client by making an authenticated connection to the Redis server. See our guide on [How to Connect to Redis and Use The Redis Database](/cloud/guides/redis-getting-started) for instructions on doing so. - -1. Switch the client to RESP3 using the following command: - - HELLO 3 - - {{< output >}} -1# "server" => "redis" -2# "version" => "6.2.6" -3# "proto" => (integer) 3 -4# "id" => (integer) 15 -5# "mode" => "standalone" -6# "role" => "master" -7# "modules" => (empty array) - {{< /output >}} - -1. Turn on client tracking for the client using the following command: - - CLIENT TRACKING on - - The client now has tracking enabled. Any data the client fetches get monitored by the server. When that data changes, the client receives a notification to invalidate its cached data. - -1. Verify that your client tracking is enabled by fetching a piece of data and making a change to it later. - - GET cat_one:key_three - - {{< output >}} -"Catharsis" - {{< /output >}} - - SET cat_one:key_three "Cliche" - - {{< output >}} --> invalidate: 'cat_one:key_three' -OK - {{< /output >}} - - {{< note respectIndent=false >}} -Redis command-line clients (using the Redis CLI) using RESP3 only receive notifications after another command has been issued by the client. - -So, for instance, a client tracking `cat_one:key_three` does not immediately receive a notification if another client changes that key. It only receives the invalidation notification whenever it attempts a command. - -Here is a breakdown of what this could look like: - -1. Client 1: `CLIENT TRACKING on` - -1. Client 1: `GET cat_one:key_three` - -1. Client 2: `SET cat_one:key_three "Climax"` - -1. Client 1: (Does not receive an immediate notification) - -1. Client 1: `GET cat_one:key_one` - - {{< output >}} --> invalidate: 'cat_one:key_three' -"Anapest" - {{< /output >}} - -{{< /note >}} - -### Using Redis's Broadcast Mode - -In its default mode, shown above, Redis's client tracking follows each key fetched by each client. If a client has not yet fetched a particular key, that client does not receive an invalidation notice when the key changes. - -But Redis's client tracking also has a broadcast mode. In this mode, a client can subscribe to a given key prefix — like `cat_one:` in the examples above. The client would then receive an invalidation notification for a change to any key with that prefix. - -Here is an example where the client turns on broadcast-mode tracking for the `cat_one:` prefix. The example uses a separate client (ID `15`) for receiving invalidation notices. - - CLIENT TRACKING on BCAST PREFIX cat_one: REDIRECT 15 - -Now, when any key with that prefix changes, an invalidation message gets sent: - - SET cat_one:key_three "Catharsis" - -{{< output >}} -1) "message" -2) "__redis__:invalidate" -3) 1) "cat_one:key_three" -{{< /output >}} - -The broadcast mode also supports multiple prefixes. The example below adds prefixes that modify the previous example: - - CLIENT TRACKING on BCAST PREFIX cat_one: PREFIX cat_two: PREFIX cat_three: REDIRECT 15 - -### Redis Client Tracking Settings - -Redis's client tracking comes with some additional parameters that let you fine-tune how the server tracks clients and sends notifications. The following sections explain what these parameters are and how you can use them. - -#### Opt-in and Opt-Out Caching - -The `OPTIN` and `OPTOUT` options let you control client tracking, by opting in or opting out, respectively, on a per-key basis. - -- Using `OPTIN`, Redis only tracks keys fetched immediately after a `CLIENT CACHING yes` command. In the example below, the `cat_one:key_one` key is not tracked but the `cat_one:key_two` key is. - - CLIENT TRACKING on OPTIN REDIRECT 15 - GET cat_one:key_one - CLIENT CACHING yes - GET cat_one:key_two - -- Using `OPTOUT`, Redis tracks as usual but allows you to use the `CLIENT CACHING no` command to exclude any key fetched immediately after from being tracked. The example below tracks `cat_one:key_one` but not `cat_one:key_two`. - - CLIENT TRACKING on OPTOUT REDIRECT 15 - GET cat_one:key_one - CLIENT CACHING no - GET cat_one:key_two - -#### No-Loop - -Some examples in the sections above demonstrated invalidation notifications for changes made by the tracked client itself. You can disable this behavior using the `NOLOOP` option for the `CLIENT TRACKING` command. - -With this option, invalidation notifications are only provided when a different client makes a change to a tracked key. - -In the example below, Redis would not provide an invalidation notice to the client. - - CLIENT TRACKING on NOLOOP REDIRECT 15 - GET cat_one:key_three - SET cat_one:key_three "Cameo" - -However, the client would receive an invalidation notice if another client executed the same `SET` command shown above. - -## Conclusion - -This tutorial has covered what you need to know to get started using Redis for server-assisted client-side caching. You learned everything from setting up client tracking to customizing it to behave the way your web application needs. - -You can continue to learn about Redis and how to get the most out of your Redis databases through our other guides in this series. These guides cover everything from [connecting to a remote Redis server](/cloud/guides/redis-getting-started) to working with the [hash data type in Redis](/cloud/guides/hashes-in-redis-databases). diff --git a/docs/guides/databases/sql-syntax/sharded-database/index.md b/docs/guides/databases/sql-syntax/sharded-database/index.md deleted file mode 100644 index 091034973ef..00000000000 --- a/docs/guides/databases/sql-syntax/sharded-database/index.md +++ /dev/null @@ -1,233 +0,0 @@ ---- -slug: sharded-database -title: "Database Sharding: Concepts, Examples, and Strategies" -description: 'Database sharding divides data into smaller chunks and distributes it across different database nodes. Learn more about sharding practices and strategies.' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2022-05-26 -modified: 2024-11-14 -keywords: ['sharded database','db sharding','sharding strategy','database sharding examples'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Wikipedia page on database sharding](https://en.wikipedia.org/wiki/Shard_(database_architecture))' -- '[MongoDB explanation of database sharding](https://www.mongodb.com/features/database-sharding-explained)' ---- - -Many software applications use a *relational database management system* (RDBMS) to store data. As the database grows, it becomes more time-and-storage intensive to store the data. One popular solution to this problem is [*database sharding*](https://en.wikipedia.org/wiki/Shard_(database_architecture)). A sharded database distributes the records in a database's tables across different databases on different computer systems. This guide explains how database sharding works and discusses some of the advantages and disadvantages of sharding. It also describes some of the main sharding strategies and provides some database sharding examples. - -## What is Database Sharding? - -As databases grow larger, they can be scaled in one of two ways. *Vertical scaling* involves upgrading the server hosting the database with more RAM, CPU ability, or disk space. This allows it to store more data and process a query more quickly and effectively. *Horizontal scaling*, which is also known as "scaling out", adds additional servers to distribute the workload. - -Data sharding is a common way of implementing horizontal scaling. Database sharding divides the table records in a database into smaller portions. Each section is a *shard*, and is stored on a different server. The database can be divided into shards based on different methods. In a simple implementation, the individual tables can be assigned to different shards. More often, the rows in a single table are divided between the shards. - -*Vertical partitioning* and *horizontal partitioning* are two different methods of partitioning tables into shards. Vertical partitioning assigns different columns within a table to different servers, but this technique is not widely used. In most cases, horizontal partitioning/sharding is used to implement sharding, and the two terms are often used interchangeably. Horizontal sharding divides the rows within a table amongst the different shards and keeps the individual table rows intact. - -{{< note >}} -Vertical partitioning and horizontal partitioning should not be confused with vertical and horizontal scaling. -{{< /note >}} - -The shards are distributed across the different servers in the cluster. Each shard has the same database schema and table definitions. This maintains consistency across the shards. Sharding allocates each row to a shard based on a sharding key. This key is typically an index or primary key from the table. A good example is a user ID column. However, it is possible to generate a sharding key from any field, or from multiple table columns. The selection of the sharding key should be reasonable for the application and effectively distribute the rows among the shards. For example, a country code or zip code is a good choice to distribute the data to geographically dispersed shards. Sharding is particularly advantageous for databases that store large amounts of data in relatively few tables, and have a high volume of reads and writes. - -Each shard can be accessed independently and does not necessarily require access to the other shards. Different tables can use different sharding techniques and not all tables necessarily have to be sharded. As an ideal, sharding strives towards a *shared-nothing* architecture, in which the shards do not share data and there is no data duplication. In practice, it is often advantageous to replicate certain data to each shard. This avoids the need to access multiple servers for a single query and can result in better performance. - -The following example demonstrates how horizontal sharding works in practice. Before the database is sharded, the example `store` table is organized in the following way: - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 1001 | Detroit | MI | 48201 | -| 1350 | Chicago | IL | 60601 | -| 2101| Cleveland | OH | 44114 | -| 2250 | Pittsburgh | PA | 15222 | -| 2455 | Boston | MA | 02108 | -| 2459 | New York | NY | 10022 | - -After sharding, one shard has half the rows from the table. - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 1001 | Detroit | MI | 48201 | -| 2101| Cleveland | OH | 44114 | -| 2455 | Boston | MA | 02108 | - -The second shard contains the remainder of the rows. - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 1350 | Chicago | IL | 60601 | -| 2250 | Pittsburgh | PA | 15222 | -| 2459 | New York | NY | 10022 | - -Sharding does not necessarily make any backup copies of the data. Each record is still only stored on a single server. *Replication* is used to copy information to another server, resulting in primary and secondary copies of the data. Replication enhances reliability and robustness at the cost of additional complexity and resources. Sharded databases can be replicated, but the procedure for doing so can be very complex. - -Replication and caching are both potential alternatives to sharding, particular in applications which mainly read data from a database. Replication spreads out the queries to multiple servers, while caching speeds up the requests. See our guide [How to Configure Source-Replica Replication in MySQL](/cloud/guides/configure-source-replica-replication-in-mysql) to learn more about data replication. - -## Pros and Cons of a Sharded Database - -Generally, a horizontal scaling approach is more robust and effective than vertical scaling. Vertical scaling is much easier to implement, because it mainly consists of hardware upgrades. It might be the correct approach to take with a medium-sized database that is slowly reaching its limit. However, it is impossible to scale any system indefinitely, and ongoing growth rapidly becomes unmanageable. The limits of vertical scaling usually lead administrators to seek another alternative. - -Horizontal scaling allows systems to achieve a much higher scaling rate. Additional servers can be added as required, permitting the database system to organically grow and access additional resources. It provides administrators with much more flexibility. - -Database sharding is a horizontal scaling strategy, so it shares the advantages of this approach. However, it also offers several additional benefits, including the following: - -- It improves performance and speeds up data retrieval. Based on the sharding key, the database system immediately knows which shard contains the data. It can quickly route the query to the right server. Because each shard only contains a subset of the rows, it is easier for the database server to find the correct entry. -- Additional computing capacity can be added with no downtime. Sharding increases the data storage capacity and the total resources available to the database. -- It can be more cost efficient to run multiple servers than one mega-server. -- Sharding can simplify upgrades, allowing one server to be upgraded at a time. -- A sharded approach is more resilient. If one of the servers is offline, the remaining shards are still accessible. Sharding can be combined with high availability techniques for even higher reliability. -- Many modern database systems provide some tools to assist with sharding, although they do not completely automate the process. - -Unfortunately, sharding also has drawbacks. Some of the downsides include: - -- Sharding greatly increases the complexity of a software development project. Additional logic is required to shard the database and properly direct queries to the correct shard. This increases development time and cost. A more elaborate network mesh is often necessary, which leads to an increase in lab and infrastructure costs. -- Latency can be higher than with a standard database design. -- [SQL join operations](/cloud/guides/sql-joins) affecting multiple shards are more difficult to execute and take longer to complete. Some operations might become too slow to be feasible. However, the right design can facilitate better performance on common queries. -- Sharding requires a lot of tuning and tweaking as the database grows. This sometimes requires a reconsideration of the entire sharding strategy and database design. Uneven shard distribution can happen even with proper planning, causing the distribution to unexpectedly become lopsided. -- It is not always obvious how many shards and servers to use, or how to choose the sharding key. Poor sharding keys can adversely affect performance or data distribution. This causes some shards to be overloaded while others are almost empty, leading to hotspots and inefficiencies. -- It is more challenging to change the database schema after sharding is implemented. It is also difficult to convert the database back to its pre-sharded state. -- Shard failures can cause cross-shard inconsistencies and other failures. -- Backup and replication tasks are more difficult with a sharded database. -- Although most RDBMS applications provide some sharding support, the tools are often not robust or complete. Most systems still do not fully support automatic sharding. - -## Database Sharding Strategies: Common Architectures - -Any sharding implementation must first decide on a db sharding strategy. Database designers must consider how many shards to use and how to distribute the data to the various servers. They must decide what queries to optimize, and how to handle joins and bulk data retrieval. A system in which the data frequently changes requires a different architecture than one that mainly handles read requests. Replication, reliability and a maintenance strategy are also important considerations. - -The choice of a sharding architecture is a critical decision, because it affects many of the other considerations. Most sharded databases have one of the following four architectures: - -- **Range Sharding**. -- **Hashed Sharding**. -- **Directory-Based Sharding**. -- **Geographic-Based Sharding**. - -### Range Sharding - -Range sharding examines the value of the sharding key and determines what range it falls into. Each range directly maps to a different shard. The sharding key should ideally be immutable. If the key changes, the shard must be recalculated and the record copied to the new shard. Otherwise, the mapping is destroyed and the location could be lost. Range sharding is also known as dynamic sharding. - -As an example, if the `userID` field is the sharding key, then records having IDs between 1 to 10000 could be stored in one shard. IDs between 10001 and 20000 map to a second shard, and those between 20001 and 30000 to a third. - -This approach is fairly easy to design and implement, and requires less programming time. The database application only has to compare the value of the sharding key to the predefined ranges using a lookup table. This scheme is also easier to redesign and maintain. Range sharding is a good choice if records with similar keys are frequently viewed together. - -Range sharding works best if there are a large number of possible values that are fairly evenly distributed across the entire range. This design works poorly if most of the key values map to the same shard. Unfortunately, this architecture is prone to poor distribution of rows among the shards. A good design can still lead to an unbalanced distribution. For example, older accounts are more likely to have been deleted over the years, leaving the corresponding shard relatively empty. This leads to inefficiencies in the database. Choosing fairly large ranges can reduce, but not eliminate, this possibility. - -The database sharding examples below demonstrate how range sharding might work using the data from the `store` database. In this case, the records for stores with store IDs under 2000 are placed in one shard. Stores possessing IDs of 2001 and greater go in the other. - -The first shard contains the following rows: - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 1001 | Detroit | MI | 48201 | -| 1350 | Chicago | IL | 60601 | - -The second shard has the following entries: - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 2101| Cleveland | OH | 44114 | -| 2250 | Pittsburgh | PA | 15222 | -| 2455 | Boston | MA | 02108 | -| 2459 | New York | NY | 10022 | - -This results in a slightly imbalanced distribution of records. However, as new stores are added, they might be assigned larger store IDs. This leads to a greater imbalance as time goes on. - -To keep the database running efficiently, shards and ranges have to be regularly rebalanced. This might involve splitting the shards apart and reassigning the data, or merging several smaller shards. If the data is not regularly monitored, performance can steadily degrade. - -### Hash Sharding (Key-Based) - -Hash-based sharding, also known as key-based or algorithmic sharding, also uses the shard key to determine which shard a record is assigned to. However, instead of mapping the key directly to a shard, it applies a hash function to the shard key. A hash function transforms one or more data points to a new value that lies within a fixed-size range. In this case, the size of the range is equal to the number of shards. The database uses the output from the hash function to allocate the record to a shard. This typically results in a more even distribution of the records to the different shards. - -This method allows multiple fields to be used as a compound shard key. This eliminates clumping and clustering, and is a better approach to use if several records can share the same key. Hash functions vary in complexity. A simple hash function calculates the remainder, or modulus, of the key divided by the number of shards. More complex hashing algorithms apply mathematically advanced equations to multiple inputs. However, it is important to use the same hash function on the same keys for each hashing operation. As with range sharding, the key value should be immutable. If it changes, the hash value must be recalculated and the database entry remapped. - -Hash sharding is more efficient than range sharding because a lookup table is not required. The hash is calculated in real time for each query. However, it is impossible to group related records together, and there is no logical connection between the records on a given shard. This requires most bulk queries to read records from multiple shards. Hash sharding is more advantageous for applications that read or write one record at a time. - -Hash sharding does not guarantee that the shards are destined to remain perfectly balanced. Patterns in the data still might lead to clustering, which can occur purely by chance. Hash sharding complicates the tasks of rebalancing and rebuilding the shards. To add more shards, it is usually necessary to re-merge all the data, recalculate the hashes, and reassign all the records. - -The following database sharding example demonstrates a simple hash sharing operation. It uses the simple hash function `store_ID % 3` to assign the records in the `store` database to one of three shards. The first step is to calculate a hash result for each entry. - -{{< note >}} -The hash results are not actually stored inside the database. They are shown in the final column for clarity. -{{< /note >}} - -| `store_ID` | `city` | `state` | `zip_code` | hash result | -|:-:|:-:|:-:|:-:|:-:| -| 1001 | Detroit | MI | 48201 | 2 -| 1350 | Chicago | IL | 60601 | 0 -| 2101| Cleveland | OH | 44114 | 1 -| 2250 | Pittsburgh | PA | 15222 | 0 -| 2455 | Boston | MA | 02108 | 1 -| 2459 | New York | NY | 10022 | 2 - -Rows having a hash result of `0` map to the first shard. - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 1350 | Chicago | IL | 60601 | -| 2250 | Pittsburgh | PA | 15222 | - -Those that have a hash result of `1` are assigned to shard number two. - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 2101| Cleveland | OH | 44114 | -| 2459 | New York | NY | 10022 | - -The remainder are stored in the third shard. - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 1001 | Detroit | MI | 48201 | -| 2459 | New York | NY | 10022 | - -In this case, although the data set is quite small, the hash function still distributes the entries evenly. This is not always the case with every database. However, as records are added, the distribution is likely to remain reasonably balanced. - -### Directory-Based Sharding - -Directory-based sharding groups related items together on the same shard. This is also known as entity or relationship-based sharding. It typically uses the value contained in a certain field to decide what shard to use. Directory sharding is accomplished through the use of a static lookup table. The table contains a list of mappings between each possible value for the field and its designated shard. Each key can only map to one shard and must appear in the lookup table exactly once. However many keys can potentially be mapped to the same shard. - -As an example, the records in a table of customers can be mapped to shards based on the customer's home state. The lookup table contains a list of all fifty states, which are the shard keys, and the shard it maps to. This allows for a system design where the records of all customers living in New England are stored on the first shard. Clients in the Mid-Atlantic are located on shard two. Clients residing in the Deep South are mapped to the third shard. - -Directory-based sharding provides a high level of control and flexibility in determining how the data is stored. When intelligently designed, it speeds up common table joins and the bulk retrieval of related data. This architecture is very helpful if the shard key can only be assigned a small number of possible values. Unfortunately, it is highly prone to clustering and imbalanced tables, and the overhead of accessing the lookup table degrades performance. However, the benefits of this architecture often outweighs its drawbacks. - -Directory-based sharding is a good choice for the `stores` database. The store entries can be distributed to the different shards based on their location. In this design, locations in New England and the mid-Atlantic are stored in the first shard, which serves as the North-East shard. Stores in the Midwest are written to the second shard. - -The first shard contains the entries displayed below. - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 2250 | Pittsburgh | PA | 15222 | -| 2455 | Boston | MA | 02108 | -| 2459 | New York | NY | 10022 | - -The second shard contains the remainder of the data. - -| `store_ID` | `city` | `state` | `zip_code` | -|:-:|:-:|:-:|:-:| -| 1001 | Detroit | MI | 48201 | -| 1350 | Chicago | IL | 60601 | -| 2101| Cleveland | OH | 44114 | - -Although these two shards are perfectly balanced, this is not the main goal of directory sharding. It instead seeks to generate useful and relevant shards of closely-related information, which this example also accomplishes. - -### Geographic-Based Sharding - -Geographic-based sharding, or *Geo-sharding*, is a specific type of directory-based sharding. Data is divided amongst the shards based on the location of the entry, which relates to the location of the server hosting the shard. The sharding key is typically a city, state, region, country, or continent. This groups geographically similar data on the same shard. It works the same way directory-based sharding does. - -A good example of geo-sharding relates to geographically dispersed customer data. The customer's home state is used as a sharding key. The lookup table maps customers living in states in the same sales region to the same shard. Each shard is located on a server located in the same region as the customer data it contains. This makes it very quick and efficient for a regional sales team to access customer data. - -## Is Sharding Right For Your Business? - -Because sharding has both advantages and drawbacks, it is important to consider which type of database benefits the most from sharding. The first part of any sharding strategy is to decide whether to shard at all. To generalize, sharding makes the most sense for a high-volume database that stores a large amount of data in a few simple tables. Sharding is especially compelling if a company expects a large increase in the size of its database. Sharding is also useful for organizations that want to access or co-locate their data on a regional basis. For instance, a large social media company would want its users to access database servers in the same country or on the same continent. This requires the company to shard its data based on user location. - -In other cases, the complexity and difficulty associated with sharding are greater than the benefits. A database with many small to medium-sized tables could use vertical scaling, increasing the storage and computing power on a single server. It could also use alternative strategies such as replication for greater resilience and read-only throughput. - -## Conclusion - -This guide answers the question, "What is database sharding?". Sharding is a method of distributing the data in a database table to several different shards based on the value of a sharding key. Each shard is stored on a different server. Ideally, the records in a sharded database are distributed amongst the shards in an equitable manner. The different shards share the same table definitions and schemas, but each record is only stored on a single shard. - -Sharding allows a database to scale horizontally, taking advantage of the increased storage, memory, and processing power that only multiple servers can offer. It also increases resiliency and performance. Each query only has to search through a portion of the total records, which is much faster. As a drawback, sharding increases the complexity of a database and increases the difficulty of joins and schema changes. - -Sharding can be accomplished using range sharding, hash sharding, or directory-based sharding. Range sharding is the easiest method, but is more likely to result in unequal shards. Hash sharding more effectively distributes the records, but is more difficult to implement. Directory-based sharding groups related items together on the same shard. - -A sharded database can be implemented using multiple Linode servers. Linode allows you to configure a full web application on a powerful Linux operating system running the industry-standard LAMP stack. Choose from a high-performance [*Dedicated CPU*](https://www.linode.com/products/dedicated-cpu/) service, or a flexible and affordable [*Shared CPU*](https://www.linode.com/products/shared/) alternative. Similarly, you can also use our [Managed Database service](https://techdocs.akamai.com/cloud-computing/docs/managed-databases) to deploy a database cluster without the need to install and maintain the database infrastructure. - -{{% content "dbass-eos" %}} \ No newline at end of file diff --git a/docs/guides/development/architectures/api-design-best-practices/index.md b/docs/guides/development/architectures/api-design-best-practices/index.md deleted file mode 100644 index f3e18723d57..00000000000 --- a/docs/guides/development/architectures/api-design-best-practices/index.md +++ /dev/null @@ -1,213 +0,0 @@ ---- -slug: api-design-best-practices -title: "API Design Best Practices: Elevate Your Development Process" -title_meta: "REST API Best Practices for Design" -description: "Searching for API design best practices to elevate your development process? We identify some of the best practices you need to follow right now. ✓ Learn more!" -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2022-03-11 -keywords: ['api design best practices','api architecture','rest api design'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Microsoft Docs: RESTful Web API Design](https://docs.microsoft.com/en-us/azure/architecture/best-practices/api-design)' -- '[freeCodeCamp: REST API Best Practices – REST Endpoint Design Examples](https://www.freecodecamp.org/news/rest-api-best-practices-rest-endpoint-design-examples/ )' -- '[Swagger: Best Practices in API Design](https://swagger.io/resources/articles/best-practices-in-api-design/ )' -- '[Stack Overflow Blog: Best Practices for REST API Design](https://stackoverflow.blog/2020/03/02/best-practices-for-rest-api-design/)' ---- - -Looking for flexibility, many modern web applications are turning to microservices and effective APIs to connect those services consistently and effectively. REST APIs in particular stand out as perhaps the most pervasive architecture for connecting microservices today. - -To learn why that is and what the REST architecture has to offer, keep reading this tutorial. Learn what goes into a REST API and the best practices to make them work effectively for you. - -## What Is REST API? - -Generally, an API (Application Programming Interface) defines a set of rules for connections between applications or services. - -REST is a particular API architecture, short for Representational State Transfer. REST APIs (also called RESTful APIs) provide flexibility through resource-oriented design and stateless communications. - -REST architectures generally adhere to the following principles: - -- Center on *resources*. A resource can be an object, data, or a service. An example of a resource that a moving company might use is a `box` object. The API provides clients with access to resources. - -- Designate *identifiers* for resources. An identifier allows a client to retrieve a particular resource from the API using a specific URI. For instance, a `box` resource with the ID `9` might be retrieved with `/boxes/9`. - -- Establish *collections*. A collection is a group of resources that may be accessible through a more general URI. Continuing the example above, a collection of all `box` resources might be found with `/boxes`. - -- Exchange in *representations* of resources. Most often, a representation uses a format like JSON or XML to hold a resource. Using such an agreed upon representational format facilitates consistent processing of resources, both by clients and by the API. - -- Employ a *stateless* communication model. The API does not store transient state data, meaning that clients can work independently with consistency. - -- Follow a uniform interface. In most cases, as you can see throughout this guide, REST APIs use the HTTP interface. Several reasons support doing so, but the main are that HTTP uses open standards and does not lock you into a particular implementation. - - Essentially, anything that can make and/or interpret HTTP requests and response can participate with a REST API using HTTP. Thus, HTTP makes RESTful APIs easier to test, extend, and scale. - -## What Are REST API Best Practices? - -RESTful best practices center on two key concepts: - -- **Maintainability**. The design should capitalize on REST's inherent flexibility, ensuing that clients and services remain independent, extendable, and adaptable. - -- **Readability**. The services should be clear so that clients can make effective use of them and developers can effectively maintain and scale them. - -What follows is a list of best practices adhering to the above concepts. Applying these REST API best design standards makes your API more effective and your development more consistent. - -### Center on Resources - -Organize your RESTful API's endpoints around resources and collections of resources. The previous section gave an example of this practice. Using a URI like `/boxes` centers the API on resources. - -For this reason, URIs should generally consist of nouns identifying resources and collections. - -This approach also favors a relational URI design. Start with a collection (e.g. `boxes`) and use an identifier to specify a particular resource. You can then access a particular collection related to that resource. - -For example, you can fetch a `box` resource with ID `9` using something like `/boxes/9`. The API can also make it possible to then fetch a collection of `content` resources associated with that particular `box`. This might be done using a URI like `/boxes/9/contents`. You could go further to fetch a particular `content` resource from that collection using its ID, `5`: `/boxes/9/contents/5`. - -### Operate via HTTP Methods - -Use the HTTP methods for actions taken on resources and collections. Doing so keeps your API focused on resources while utilizing a standard approach to define operations related to those resources. - -To illustrate what this might look like, take a look at the following table: - -| URI | GET | POST | PUT | DELETE | -| :------------: | :----------: | :----------: | :----------: | :----------: | -| /boxes | Fetch all boxes | Create a new box | Update all boxes | Delete all boxes | -| /boxes/9 | Fetch box ID 9 | N/A | Update box ID 9 | Delete box ID 9 | -| /boxes/9/contents | Fetch the contents of box ID 9 | Create a new content resource for box ID 9 | Update all contents for box ID 9 | Delete all contents from box ID 9 | - -This way, your API is able to handle all actions related to fetching, creating, and modifying resources without any additional endpoints. Using HTTP standards for actions makes it easier for developers to anticipate how to take particular actions on resources, without having to research specific documentation. - -For this reason, it's a good idea to avoid using verbs in URIs wherever feasible. This keeps your API consistent and predictable. If you need to use verbs for a URI, use them like a resource name. For example, to process a `box` resource with ID `9` for shipping, you might use `/boxes/9/send`. - -### Limit Results - -Make resource fetching more efficient by implementing pagination, filtering, sorting, and searching options for your REST API. Doing so lets clients fetch only the resources they need, which, in turn, improves network traffic. - -For instance, say a client intends to display a list of boxes. Good UI design typically has long lists divided into pages, so this client is taking that approach. Retrieving all of the boxes at once unnecessarily extends the load time, potentially degrading user experience. - -So, the API provides options that enable pagination. The client can thus use query string parameters to get one page of boxes at a time. This example assumes a page would have ten boxes listed and that the client is loading the second page: `/boxes?limit=10&page=2`. - -The same logic applies for filtering and searching. REST APIs should implement query strings on collection endpoints. This allows clients to limit responses to only what is needed. In this next example, the client would be able to fetch only the most recent boxes that are also in `pending` status: `/boxes?mostRecent=true&status=pending`. - -### Name Consistently - -Use consistent naming conventions for URIs. This means using the same conventions for all resources and collections. You may use a separate convention for parameters, like those used in query strings, but you need to adhere to it consistently as well. - -What follows are some particular recommendations based on general naming standards. These are not absolute rules. However, following these conventions can make your APIs easier for developers to adapt to, since they are common in the web development field. - -- Use single words for resource and collection names. Ensure that collection names are plurals of the resources they group, as in `boxes` for a collection of `box` resources. - -- Should you need more than one word for a resource/collection name, use "kebab case". Thus, a collection of herbal teas would be `herbal-teas`. - -- Should you need more than one word for parameters, use "camel case". For example, use `minValue` for a parameter giving a minimum value. This practice applies to both query string parameters and parameters housed in resource objects. The practice matches conventions for JSON design, the most frequently used representational format for REST APIs. - -### Employ Versioning - -Version your RESTful API when significant changes take effect that may impact clients. Requirements change, and one of the perks of REST APIs is adaptability. However, you don't want to negatively impact clients that already have expectations from the API. - -Versioning allows clients to continue accessing API endpoints as they were at a given point in time. At the same time, it allows you to continue moving the API forward for new clients and requirements. - -Three main approaches exist for versioning your REST APIs. - -- **URI Versioning**: This adds a version indicator to the URIs for your REST APIs. Commonly, APIs employ a simple version indicator at the beginning of the URI path. For instance, a client looking to fetch `box` ID `9` using the first version of the API might use `/v1/boxes/9`. - -- **Query-String Versioning**: This gives clients the option of indicating the desired version in a query string when making requests. One benefit of this approach is that the query string can be optional, with a default API version used when the query string is omitted. Here is an example of what a client might call to fetch `box` ID `9` from version `2` of the API: `/boxes/9?apiVersion=2`. - -- **Header Versioning**: This option works similarly to query-string versioning, but hides the process in the headers. Like query-string versioning, it has the advantage of being able to provide a default version. To give an example, here is a request to fetch `box` ID `9` from version `3` of the API: - - ```command - GET /boxes/9 - Custom-Header: api-version=3 - ``` - -Keep in mind that both header versioning and query-string versioning require the API to implement additional logic for processing versions. - -### Maintain Documentation - -Document your REST API. With proper documentation, you help developers quickly, efficiently, and accurately make use of endpoints. Ultimately, it saves significant time keeping developers on the same page and preventing duplicate work. - -Effective API documentation should include: - -- A list of endpoints. - -- A description for each endpoint. - -- An example request and response for each end point. - -Preferably, readers should also be able to test the requests right there in the documentation. Many good examples of REST API documentation allow readers to modify a base request object and immediately test the result in a web browser. - -Numerous tools exist to help with documenting REST APIs. Often, these can automatically generate a list of endpoints and basic request and response skeletons, easing the documentation effort significantly. They may also have built-in tools for letting users test API responses. Of these tools, [Swagger](https://swagger.io/) is perhaps the most popular and has seen the most widespread usage in recent years. - -### Implement Security - -Secure your RESTful API using SSL and authentication tokens. Using SSL protects API connections from attacks, while authentication allows you to ensure that only authorized users have access. - -To learn more about SSL certification, check out our guides [Understanding TLS Certificates and Connections](/cloud/guides/what-is-a-tls-certificate) and [Securing Web Traffic Using Certbot](/cloud/guides/enabling-https-using-certbot). - -Recall that REST APIs are stateless. Thus, the preferred path for authentication on REST APIs is through the use of authentication tokens. In this scenario, a client may post credentials to a given endpoint. The API validates the credentials and assigns the user a random token in response. The client must then include that token in its requests to other endpoints. - -You can find information on the implementation of JSON Web Tokens, for example, in our guide [User Authentication with JSON Web Tokens (JWTs) and Express](/cloud/guides/how-to-authenticate-using-jwt). - -### Respond with Statuses - -Provide clear status and error codes using HTTP conventions. These provide a resource for clients to quickly and consistently interpret responses. Using the accepted HTTP conventions here carries the same benefits as using HTTP generally. Almost any client capable of sending and receiving HTTP requests and responses has the tools to interpret these codes. - -Doing so also means gracefully handling errors. When encountering an error processing a request, REST APIs should provide a response with an appropriate error code. - -To get you started, here's a broad overview of the HTTP status codes: - -| Code Range | Description | -| :---------: | :---------: | -| 100–199 | Informational responses | -| 200–299 | Success responses | -| 300–399 | Responses for redirects | -| 400–499 | Client-side error responses | -| 500–599 | Server-side error responses | - -Most REST APIs only use codes in the 200, 400, and 500 areas. Of these, here are some of the most common and useful: - -- `200` indicates a successful request that does not result in a resource being created. So you can typically see this code in response to things like `GET` and `PUT` requests. - -- `201` indicates a successful request in which a resource has been created. This is typically the response code for `POST` requests, although some `POST` requests do not result in the creation of a resource. - -- `400` indicates an error in the request, typically with the request body. This code can be used to indicate when the API was unable to parse the request body. - -- `401` indicates that the request did not provide appropriate authentication. This can be the case when the client omits authentication or uses an outdated authentication token. - -- `403` indicates that an otherwise authenticated client did not have access to the requested API or resource. - -- `404` indicates that the requested URI does not correspond to an API endpoint. - -- `500` indicates an internal server error. Typically, this response is provided when the server encounters an unknown error. - -You can find a more complete list of HTTP status codes, along with descriptions of each, in Mozilla's [HTTP response status codes](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) documentation. - -### Include Related URIs - -Consider using *HATEOAS* or "Hypermedia as the Engine of Application State". This dictates that RESTful API usage should be based on response metadata rather than outside knowledge. This works by each request providing URIs for related request endpoints. - -For example, the `GET` response for the `box` resource with ID `9` could include URIs indicating how a client could operate on the resource: - -```file {lang="json"} -{ - "id": 9, - "numberOfContainedItems": 6, - "status": "pending", - "links": [ - { - "method": "PUT", - "action": "ship", - "uri": "/boxes/9/send" - }, - { - "method": "PUT", - "action": "receive", - "uri": "/boxes/9/receive" - } - ] -} -``` - -Following HATEOAS can make your services more adaptable to future changes. For one, it conveniently informs clients of available related endpoints. More importantly, URIs supplied in response add flexibility. Should a URI have to change, like from `/send` to `/ship` in the example above, the client does not have to modify its approach. It simply fetches the necessary URI from the response. - -## Conclusion - -REST APIs bring flexibility and consistency to your microservices, provided you implement them with best practices in mind. This tutorial walked you through standards you can apply when designing and implementing your own REST APIs. These principles can help you create and maintain efficient and manageable RESTful APIs. \ No newline at end of file diff --git a/docs/guides/development/ci/use-buildbot-for-software-testing-on-ubuntu/index.md b/docs/guides/development/ci/use-buildbot-for-software-testing-on-ubuntu/index.md deleted file mode 100644 index 457e9720089..00000000000 --- a/docs/guides/development/ci/use-buildbot-for-software-testing-on-ubuntu/index.md +++ /dev/null @@ -1,442 +0,0 @@ ---- -slug: use-buildbot-for-software-testing-on-ubuntu -title: "Using Buildbot to Test Software on Ubuntu 18.04" -title_meta: "Use Buildbot for Software Testing on Ubuntu 18.04" -description: "This shows how to install, configure, and use Buildbot as a continuous integration platform, as well as how to write configuration for custom testing builds." -og_description: "Deploy self-hosted continuous integration using Buildbot." -authors: ["Tyler Langlois"] -contributors: ["Tyler Langlois"] -published: 2018-09-06 -modified: 2021-10-18 -keywords: ["buildbot", "testing", "python", "continuous integration", "ci", "build", "qa"] -tags: ["automation","nginx"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Official Buildbot Tutorial](http://docs.buildbot.net/current/tutorial/)' -- '[Buildbot Documentation](http://docs.buildbot.net/current/index.html)' -audiences: ["intermediate"] -dedicated_cpu_link: true -relations: - platform: - key: buildbot-software-testing - keywords: - - distribution: Ubuntu 18.04 -aliases: [] ---- - -[Buildbot](https://buildbot.net/) is an open source system for testing software projects. In this guide, you will set up a Linode as a Buildbot server to use as a continuous integration platform to test code. Similarly to hosted solutions like Travis CI, Buildbot is an automated testing platform that can watch for code changes, test a project's code, and send notifications regarding build failures. - -## Before you Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -3. Complete the [Add DNS Records](/cloud/guides/set-up-web-server-host-website#add-dns-records) steps to register a domain name that will point to your Linode instance hosting Buildbot. - - {{< note respectIndent=false >}} -Replace each instance of `example.com` in this guide with your Buildbot site's domain name. -{{< /note >}} - -1. Your Buildbot site will serve its content over HTTPS, so you will need to obtain an SSL/TLS certificate. Use [Certbot](/cloud/guides/secure-http-traffic-certbot#use-certbot-on-ubuntu) to request and download a free certificate from [Let's Encrypt](https://letsencrypt.org/). - - sudo apt install software-properties-common - sudo add-apt-repository ppa:certbot/certbot - sudo apt update - sudo apt install certbot - sudo certbot certonly --standalone -d example.com - - These commands will download a certificate to `/etc/letsencrypt/live/example.com/` on your Linode. - - {{< note respectIndent=false >}} - The steps to install NGINX will be covered in the [Set up the Buildbot Master Web Interface](/cloud/guides/use-buildbot-for-software-testing-on-ubuntu#set-up-the-buildbot-master-web-interface) section of the guide. - {{< /note >}} - -## Install Buildbot - -### Install the Buildbot Master - -Since Buildbot is provided as an Ubuntu package, install the software from the official Ubuntu repositories. - -1. Install the `buildbot` package along with `pip3`, which will be used to install additional python packages: - - sudo apt-get install -y buildbot python3-pip - -1. Install the required Buildbot Python packages: - - sudo pip3 install buildbot-www buildbot-waterfall-view buildbot-console-view buildbot-grid-view - -1. The `buildbot` package sets up several file paths and services to run persistently on your host. In order to create a new configuration for a Buildbot master, enter the directory for Buildbot master configurations and create a new master called `ci` (for "continuous integration"). - - cd /var/lib/buildbot/masters - sudo -u buildbot -- buildbot create-master ci - - The generated master configuration file's location is `/var/lib/buildbot/masters/ci/master.cfg.sample`. - -1. Make a copy of the default configuration to the path that Buildbot expects for its configuration file: - - sudo cp ci/master.cfg.sample ci/master.cfg - -1. Change the permissions for this configuration file so that the `buildbot` user has rights for the configuration file: - - sudo chown buildbot:buildbot ci/master.cfg - -### Configure the Buildbot Master - -In order to secure and customize Buildbot, you will change a few settings in the master configuration file before using the application. The master configuration file's location is `/var/lib/buildbot/masters/ci/master.cfg`. - -Buildbot has a number of concepts that are represented in the master build configuration file. Open this file in your preferred text editor and browse the Buildbot configuration. The Buildbot configuration is written in Python instead of a markup language like Yaml. - -1. Generate a random string to serve as the password that workers will use to authenticate against the Buildbot master. This is accomplished by using `openssl` to create a random sequence of characters. - - openssl rand -hex 16 - - -1. Update the following line in the `master.cfg` file and replace `pass` with the randomly-generated password: - - {{< file "/var/lib/buildbot/masters/ci/master.cfg" python >}} -... -# The 'workers' list defines the set of recognized workers. Each element is -# a Worker object, specifying a unique worker name and password. The same -# worker name and password must be configured on the worker. -c['workers'] = [worker.Worker("example-worker", "pass")] -... - {{}} - -1. Uncomment the `c[title]` and the `c[titleURL]` lines. If desired, change the name of the Buildbot installation by updating the value of ``c[title]``. Replace the `c[titleURL]` value with the URL of your Buildbot instance. In the example, the URL value is replaced with `example.com`. - - {{< file "/var/lib/buildbot/masters/ci/master.cfg" python >}} -... -c['title'] = "My CI" -c['titleURL'] = "https://example.com" -... - {{}} - -1. Uncomment the `c['buildbotURL']` line and replace the URL value with the your Buildbot instance's URL: - - {{< file "/var/lib/buildbot/masters/ci/master.cfg" python >}} -... -c['buildbotURL'] = "https://example.com/" -... - {{}} - - These options assume that you will use a custom domain secured with Let's Encrypt certificates from `certbot` as outlined in the [Before You Begin](/cloud/guides/use-buildbot-for-software-testing-on-ubuntu#before-you-begin) section of this guide. - -1. Uncomment the web interface configuration lines and keep the default options: - - {{< file "/var/lib/buildbot/masters/ci/master.cfg" python >}} -... -c['www'] = dict(port=8010, - plugins=dict(waterfall_view={}, console_view={}, grid_view={})) -... - {{}} - -1. By default, Buildbot does not require people to authenticate in order to access control features in the web UI. To secure Buildbot, you will need to configure an authentication plugin. - - Configure users for the Buildbot master web interface. Add the following lines below the web interface configuration lines and replace the `myusername` and `password` values with the ones you would like to use. - - {{< file "/var/lib/buildbot/masters/ci/master.cfg" python >}} -... -c['www'] = dict(port=8010, - plugins=dict(waterfall_view={}, console_view={}, grid_view={})) - -# user configurations -c['www']['authz'] = util.Authz( - allowRules = [ - util.AnyEndpointMatcher(role="admins") - ], - roleMatchers = [ - util.RolesFromUsername(roles=['admins'], usernames=['myusername']) - ] -) -c['www']['auth'] = util.UserPasswordAuth([('myusername','password')]) -... - {{}} - -1. Buildbot supports building repositories based on GitHub activity. This is done with a GitHub webhook. Generate a random string to serve as a webhook secret token to validate payloads. - - openssl rand -hex 16 - - -1. Configure Buildbot to recognize GitHub webhooks as a change source. Add the following snippet to the end of the `master.cfg` file and replace `webhook secret` with the random string generated in the previous step. - - {{< file "/var/lib/buildbot/masters/ci/master.cfg" python >}} -c['www']['change_hook_dialects'] = { - 'github': { - 'secret': 'webhook_secret', - } -} - {{}} - -1. Finally, start the Buildbot master. This command will start the Buildbot process and persist it across reboots. - - sudo systemctl enable --now buildmaster@ci.service - -### Set up the Buildbot Master Web Interface - -Buildbot is now running and listening on HTTP without encryption. To secure the connection, install NGINX to terminate SSL and reverse proxy traffic to the Buildbot master process. - -{{% content "install-nginx-ubuntu-ppa" %}} - -Now that NGINX is installed, configure NGINX to talk to the local Buildbot port. NGINX will listen for SSL traffic using the Let's Encrypt certificate for your domain. - -1. Create your site's NGINX configuration file. Ensure that you replace the configuration file's name `example.com.conf` with your domain name. Replace all instances of `example.com` with your Buildbot instance's URL. - - {{< file "/etc/nginx/conf.d/example.com.conf" conf >}} -server { - # Enable SSL and http2 - listen 443 ssl http2 default_server; - - server_name example.com; - - root html; - index index.html index.htm; - - ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; - - # put a one day session timeout for websockets to stay longer - ssl_session_cache shared:SSL:10m; - ssl_session_timeout 1440m; - - ssl_protocols TLSv1.2 TLSv1.3; - ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384; - ssl_prefer_server_ciphers on; - - # force https - add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; - spdy_headers_comp 5; - - proxy_set_header HOST $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Server $host; - proxy_set_header X-Forwarded-Host $host; - - location / { - proxy_pass http://127.0.0.1:8010/; - } - location /sse/ { - # proxy buffering will prevent sse to work - proxy_buffering off; - proxy_pass http://127.0.0.1:8010/sse/; - } - location /ws { - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "upgrade"; - proxy_pass http://127.0.0.1:8010/ws; - # raise the proxy timeout for the websocket - proxy_read_timeout 6000s; - } -} -{{< /file >}} - -1. Disable NGINX's default configuration file: - - mv /etc/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf.disabled - -1. Restart NGINX to apply the Buildbot reverse proxy configuration: - - sudo systemctl restart nginx - -1. Navigate to your Buildbot instance's URL over HTTPS. You will see the Buildbot homepage: - - ![Buildbot Landing Page](buildbot-landing-page.png "Buildbot Landing Page") - - Your continuous integration test server is now up and running. - -1. Ensure that you can log into your Buildbot instance with the admin credentials you created in the [Configure Buildbot Master](/cloud/guides/use-buildbot-for-software-testing-on-ubuntu#set-up-the-buildbot-master-web-interface) section. Click on the top right hand dropdown menu entitled **Anonymous** and then, click on **Login**. A *Sign In* modal will appear. Enter your credentials to log in to Buildbot as the admin user. - -### Install the Buildbot Worker - -In order for Buildbot to execute test builds, the Buildbot master will require a worker. The following steps will setup a worker on the same host as the master. - -1. Install the `buildbot-worker` using pip: - - pip install buildbot-worker - -1. Navigate to the directory which will store the Buildbot worker configurations: - - cd /var/lib/buildbot/workers - -1. Create the configuration directory for the Buildbot worker. Replace `example-worker` and `my-worker-password` with the values used for the `c[worker]` configuration in the `master.cfg` file. - - sudo -u buildbot -- buildbot-worker create-worker default localhost example-worker my-worker-password - -1. The Buildbot worker is ready to connect to the Buildbot master. Enable the worker process. - - sudo systemctl enable --now buildbot-worker@default.service - - Confirm that the worker has connected by going to your Buildbot site and navigating to **Builds** -> **Workers** in the sidebar menu: - - ![Buildbot Workers Page](buildbot-worker-page.png "Buildbot Workers Page") - -## Configuring Builds - -Now that Buildbot is installed, you can configure it to run builds. In this tutorial, we will use a forked GitHub repository for the [Linode Guides and Tutorials repository](https://github.com/linode/docs) to illustrate how to use Buildbot as a system to run tests against a repository. - -### Configuring GitHub - -Before creating the build configuration, fork the `linode/docs` repository into your GitHub account. This is the repository that will be used to run tests against. The repository will also require webhooks to be configured to send push or PR events to Buildbot. - -{{< note >}} -The actions you take to fork, add webhook, and push changes to your fork of `linode/docs` will not affect the parent (or upstream), so you can safely experiment with it. Any changes you make to branches of your fork will remain separate until you submit a pull request to the original `linode/docs` repository. -{{< /note >}} - -#### Forking and Configuring the Repository - -1. Log in to your GitHub account and navigate to https://github.com/linode/docs. Click the **Fork** button: - - ![GitHub Fork Button](github-fork-button.png "GitHub Fork Button") - -1. Choose the account to fork the repository into (typically just your username). GitHub will bring you to the page for your own fork of the `linode/docs` repository. - - Select **Settings** to browse your fork's settings: - - ![GitHub Fork Settings](github-fork-settings.png "GitHub Fork Settings") - - Then, select **Webhooks** from the sidebar: - - ![GitHub Webhook Settings](github-webhook-settings.png "GitHub Webhook Settings") - -1. Click on the **Add webhook** button. There are several fields to populate: - - * Under *Payload URL* enter the domain name for your Buildbot server with the change hook URL path appended to it: `https://example.com/change_hook/github`. - * Leave the default value for *Content type*: `application/x-www-form-urlencoded`. - * Under the *Secret* field, enter the `secret` value for the `c['www']['change_hook_dialects']` option you configure in the `master.cfg` file. - * Leave *Enable SSL Verification* selected. - * For the *Which events would you like to trigger this webhook?*, select *Let me select individual events* and ensure that only the following boxes are checked: - * *Pull requests* - * *Pushes* - * Leave *Active* selected to indicate that GitHub should be configured to send webhooks to Buildbot. - -1. Click on the **Add webhook** button to save your settings. - - GitHub will return your browser to the list of webhooks for your repository. After configuring a new webhook, GitHub will send a test webhook to the configured payload URL. To indicate whether GitHub was able to send a webhook without errors, it adds a checkmark to the webhook item: - - ![GitHub Webhook Success](github-webhook-success.png "GitHub Webhook Success") - - Github will now send any new pushes made to your fork to your instance of Buildbot for testing. - -### Build Prerequisites - -This guide runs builds as a simple process on the Buildbot worker, however, it is possible to execute builds within a Docker container, if desired. Consult the official [Buildbot documentation](http://docs.buildbot.net/current/manual/cfg-workers-docker.html) for more information on configuring a Docker set up. - -Most software projects will define several prerequisites and tests for a project build. The Linode Guides and Tutorials repository defines several different tests to run for each build. This example will use one test defined in a python script named `blueberry.py`. This test checks for broken links, missing images, and more. This test's dependencies can be installed via `pip` in a virtualenv. - -On your Linode, install the packages necessary to permit the worker to use a Python virtualenv to create a sandbox during the build. - - sudo apt-get install -y build-essential python3-dev python3-venv - -### Writing Builds - -The `/var/lib/buildbot/masters/ci/master.cfg` file contains options to configure builds. The specific sections in the file that include these configurations are the following: - -* `WORKERS`, define the worker executors the master will connect to in order to run builds. -* `SCHEDULERS`, specify how to react to incoming changes. -* `BUILDERS`, outline the steps and build tests to run. - -Because the worker has already been configured and connected to the Buildbot master, the only settings necessary to define a custom build are the `SCHEDULERS` and `BUILDERS`. - -1. Add the following lines to the end of the `/var/lib/buildbot/masters/ci/master.cfg` file to define the custom build. Ensure you replace `my-username` and `my-git-repo-name` with the values for your own GitHub fork of the `linode/docs` repository and `example-worker` with the name of your Buildbot instance's worker: - - {{< file "/var/lib/buildbot/masters/ci/master.cfg" python >}} -docs_blueberry_test = util.BuildFactory() -# Clone the repository -docs_blueberry_test.addStep( - steps.Git( - repourl='git://github.com/my-username/my-git-repo-name.git', - mode='incremental')) -# Create virtualenv -docs_blueberry_test.addStep( - steps.ShellCommand( - command=["python3", "-m", "venv", ".venv"])) -# Install test dependencies -docs_blueberry_test.addStep( - steps.ShellCommand( - command=["./.venv/bin/pip", "install", "-r", "ci/requirements.txt"])) -# Run tests -docs_blueberry_test.addStep( - steps.ShellCommand( - command=["./.venv/bin/python3", "ci/blueberry.py"])) -# Add the BuildFactory configuration to the master -c['builders'].append( - util.BuilderConfig(name="linode-docs", - workernames=["example-worker"], - factory=docs_blueberry_test)) - {{< /file >}} - - The configuration code does the following: - - * A new Build Factory is instantiated. Build Factories define how builds are run. - * Then, instructions are added to the Build Factory. The Build Factory clones the GitHub fork of the `linode/docs` repository. - * Next, a Python virtualenv is setup. This ensures that the dependencies and libraries used for testing are kept separate, in a dedicated sandbox, from the Python libraries on the worker machine. - * The necessary Python packages used in testing are then installed into the build's virtualenv. - * Finally, the `blueberry.py` testing script is run using the `python3` executable from the virtualenv sandbox. - * The defined Build Factory is then added to the configuration for the master. - -1. Define a simple scheduler to build any branch that is pushed to the GitHub repository. Add the following lines to the end of the `master.cfg` file: - - {{< file "~/buildbox-master/master/master.cfg" python >}} - ... -c['schedulers'].append(schedulers.AnyBranchScheduler( - name="build-docs", - builderNames=["linode-docs"])) - {{< /file >}} - - This code instructs the Buildbot master to create a scheduler that builds any branch for the `linode-docs` builder. This scheduler will be invoked by the change hook defined for GitHub, which is triggered by the GitHub webhook configured in the GitHub interface. - -1. Restart the Buildbot master now that the custom scheduler and builder have been defined: - - sudo systemctl restart buildmaster@ci.service - -### Running Builds - -Navigate to your Buildbot site to view the Builder and Scheduler created in the previous section. In the sidebar click on **Build** -> **Builders**. You will see `linode-docs` listed under the *Builder Name* heading: - -![Buildbot Custom Builder](buildbot-custom-builder-page.png "Buildbot Custom Builder") - -A new build can be started for the `linode-docs` builder. Recall that the GitHub webhook configuration for your fork of `linode/docs` is set to call Buildbot upon any push or pull request event. To demonstrate how this works: - -1. Clone your fork of the `linode/docs` repository on your local machine (do not run the following commands on your Buildbot server) and navigate into the cloned repository. Replace `username` and `repository` with your own fork's values: - - git clone https://github.com/username/repository.git - cd repository - -1. Like many git repositories, the `linode-docs` repository changes often. To ensure that the remaining instructions work as expected, start at a specific revision in the code that is in a known state. Check out revision `76cd31a5271b41ff5a80dee2137dcb5e76296b93`: - - git checkout 76cd31a5271b41ff5a80dee2137dcb5e76296b93 - -1. Create a branch starting at this revision, which is where you will create dummy commits to test your Buildbot master: - - git checkout -b linode-tutorial-demo - -1. Create an empty commit so that you have something to push to your fork: - - git commit --allow-empty -m 'Buildbot test' - -1. Push your branch to your forked remote GitHub repository: - - git push --set-upstream origin linode-tutorial-demo - -1. Navigate to your Buildbot site and go to your running builds. The **Home** button on the sidebar displays currently executing builds. - - ![Buildbot running Builds](buildbot-running-builds.png "Buildbot running Builds") - -1. Click on the running build to view more details. The build will display each step along with logging output: - - ![Buildbot Build Page](buildbot-build-page.png "Buildbot Build Page") - - Each step of the build process can be followed as the build progresses. While the build is running, click on a step to view standard output logs. A successful build will complete each step with an exit code of `0`. - - Your Buildbot host will now actively build pushes to any branch or any pull requests to your repository. - -## Features to Explore - -Now that you have a simple build configuration for your Buildbot instance, you can continue to add features to your CI server. Some useful functions that Buildbot supports include: - -* [Reporters](http://docs.buildbot.net/current/manual/cfg-reporters.html), which can notify you about build failures over IRC, GitHub comments, or email. -* [Workers](http://docs.buildbot.net/current/manual/cfg-workers.html) that execute builds in Docker containers or in temporary cloud instances instead of static hosts. -* [Web server features](http://docs.buildbot.net/current/manual/cfg-www.html), including the ability to generate badges for your repository indicating the current build status of the project. diff --git a/docs/guides/development/concepts/introduction-to-the-solid-data-protocol/index.md b/docs/guides/development/concepts/introduction-to-the-solid-data-protocol/index.md deleted file mode 100644 index 91767599bad..00000000000 --- a/docs/guides/development/concepts/introduction-to-the-solid-data-protocol/index.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -slug: introduction-to-the-solid-data-protocol -title: "An Overview of the Solid Data Protocol" -title_meta: "Introduction to the Solid Data Protocol" -description: 'This guide will introduce you to the Solid protocol, created by Tim Berners-Lee and gives Internet users ownership over their data, and how to incorporate the tech.' -authors: ["Andy Patrizio"] -contributors: ["Andy Patrizio"] -published: 2021-08-27 -keywords: ['solid protocol'] -tags: ['web applications'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Solid Getting Started Developer Guide](https://solidproject.org/developers/tutorials/getting-started)' -- '[Solid GitHub page](https://github.com/solid/solid-tutorial-intro)' ---- -## The Origins of the Solid Protocol - -Just over thirty years ago, a physicist named [Tim Berners-Lee](https://en.wikipedia.org/wiki/Tim_Berners-Lee) created a decentralized, collaborative read-write space he called the World Wide Web. Initially, the first browser was also an editor for content creation. As the web grew, web content shifted to development and design tools. HTML made up less of a web page as cascading style sheets, JavaScript, Python, and other technologies took over. As this happened, user data was taken out of the domain of its owner and transferred to the Internet companies that own the websites. It’s been the cause for debate for some time; do you own the contents of your Facebook page or does Facebook? - -Another issue is the transferability and mobility of data. For the longest time, we kept our data on our PC. Then came multiple PCs, phones and tablets, and cloud storage. Now your personal data is scattered across several sites and their own storage implementations. Not just that, but your data is difficult to migrate since different apps store data very differently. - -Dr. Berners-Lee has become increasingly disillusioned with how the web has been misused to violate the privacy of its users. For this reason, he sought to create a new means to protect an individual's personal data. With a donation from Mastercard in 2015, he and his research team at the Massachusetts Institute of Technology began the work on *Solid* (“social linked data”). In 2018, he took a sabbatical from MIT to launch a commercial venture based on Solid, called *Inrupt*. The company's mission is to provide an ecosystem to help protect the integrity and quality of the new web built on Solid. It’s a way for you to own your own data while making it available to the applications that you want to allow to use it. - -## What is the Solid Protocol? - -[Solid](https://solidproject.org/) is a tech stack that incorporates a group of related protocols and implementations, much like the web Berners-Lee created. All of the protocols used are W3C-approved standards, and some were even developed by Berners-Lee himself. - -Solid is a decentralized platform for social web applications. With this new design, a user’s data is managed independently of the applications that create and consume this data. Solid uses *Pods* to store your data. It is possible to have multiple Pods, and they can be stored anywhere; on your PC, mobile devices, or in the cloud. You then have control over which apps and individuals may have access to a specific bit of data within your Pod. You can also revoke access at any given time. - -The protocols used by Solid are based on existing W3C recommendations for reading, writing, and access control of the contents of users’ Pods. In Solid architecture, applications can operate over data owned by the user or that the user has access to. The storage location of the data on the web does not influence how applications can interact with your data. The app becomes irrelevant in terms of accessing the data so long as it uses the W3C authentication and access standards. Any app can access any Pod. - -## The Solid Protocol and Medical Records - -The Solid protocol has the potential to make a big impact on the security of your medical records. The portability of medical records and medical history is something the medical industry has pursued for years. Medical records are kept in silos with no connection between each record. Solid allows a person to create a Pod where only their medical records are stored. A user can then give access to their medical records to their physicians and turn it off if they change doctors. - -## How Solid Works - -Solid applications are implemented as client-side web or mobile applications that read and write data directly from the Pods. Applications are by design decoupled from the data source. This allows applications to aggregate data from different sources on the web. The application can access both the user’s Pod and other Pods, and multiple applications can reuse the same data on a Pod. - -Berners-Lee explained that Solid is not unlike a typical web app. You have a front-end (the app or web page) and a back-end (a server app). Both pieces communicate using a custom server API. With Solid, all users have a universal API that handles all the backend data and access control. - -Solid uses decentralized authentication, a global ID space, and global single sign-on. Solid uses the WebID identifier and protocol that Berners-Lee invented in 2000 for user access, along with a cryptographic key. - -Solid applications read and write data stored in users’ Pods via RESTful HTTP operations using Linked Data Platform (LDP). Pods use LDP to organize data into containers that group resources together, giving each container and resource their own Uniform Resource Identifier (URI). - -The Solid platform includes solid.js, a JavaScript library implementing the Solid protocols. The Solid protocols guarantee efficient performance for social applications regardless if these applications use solid.js or not. Usage of solid.js is intended to accelerate the development of Solid applications by enabling you to write less code. - -For more advanced options, Solid supports [SPARQL](https://en.wikipedia.org/wiki/SPARQL). SPARQL is an Resource Description Framework (RDF) query language able to retrieve and manipulate data stored in RDF format. It allows applications to express complex data retrieval operations, including operations that require server-to-server communication via link-following. This simplifies Solid application development since it enables a developer to delegate complex, multi-pod data retrieval operations to the server. - -Berners-Lee has been working on Solid since 2016, but it is still in its early stages and the community is still growing. The [subreddit for Solid](https://www.reddit.com/r/SOLID/) currently has about 2,000 members. If you'd like to learn more, the project has a [home page with documentation](https://solidproject.org/developers/tutorials/getting-started) on how to build Solid apps, and there is a [GitHub page](https://github.com/solid/solid-tutorial-intro) with sample code and tutorials as well. diff --git a/docs/guides/development/concepts/oop-principles/index.md b/docs/guides/development/concepts/oop-principles/index.md deleted file mode 100644 index 2598884d201..00000000000 --- a/docs/guides/development/concepts/oop-principles/index.md +++ /dev/null @@ -1,314 +0,0 @@ ---- -slug: oop-principles -title: "A Guide to OOP Principles in Java" -title_meta: "Object-Oriented Programming Concepts, Explained" -description: "What are the four major OOP principles? This guide will discuss object-oriented programming concepts and provide real-world examples." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2022-06-10 -keywords: ['oop principles','oop concepts','oop concepts in java'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[GeeksforGeeks: Object Oriented Programming (OOPs) Concept in Java](https://www.geeksforgeeks.org/object-oriented-programming-oops-concept-in-java/)' -- '[ParTech: Basic Principles of Object-oriented Programming](https://www.partech.nl/en/publications/2020/10/basic-principles-of-object-oriented-programming#:~:text=The%20basic%20principles%20of%20OOP,any%20object%2Doriented%20programming%20language.)' -- '[Munish Chandel: What are Four Basic Principles of Object Oriented Programming?](https://medium.com/@cancerian0684/what-are-four-basic-principles-of-object-oriented-programming-645af8b43727)' -- '[Key Lime Interactive: The Four Pillars of Object Oriented Programming](https://info.keylimeinteractive.com/the-four-pillars-of-object-oriented-programming)' ---- - -Object-oriented programming gives you a set of programming principles to make your code more compartmentalized and reusable. Object-oriented programming accomplishes this by structuring programs around objects. This tutorial covers the core principles of object-oriented programming and provides examples of these concepts written in Java. - -## What is Object Oriented Programming (OOP)? - -Object-oriented programming — often abbreviated "OOP" — is a set of programming principles centered on objects. Such a set of principles is called a programming *paradigm*. Objects in OOP can hold attributes and be assigned behaviors, and they allow developers to structure programs around reusable, self-contained components. - -Because of its object-oriented focus, OOP shines when used for applications that need objects to be at their logical center. This is the case with user interfaces, one of the most common places to see OOP used, as well as business applications. - -## OOP Concepts in Java You Need to Know - -Object-oriented programming tends to make use of four structures. These form the bedrock of all of the pieces a developer has to work with when building object-oriented programs. - -- **Classes**: These act as blueprints for objects. They define underlying properties and behaviors which can be inherited by other classes and by objects. Your OOP program's collection of classes creates a structure off of which the rest of the program gets built. - - Often, in Java, code is constructed with one class per file. The class syntax resembles the following example: - - {{< file >}} -public class ClassName { - // Code related to the class. -} - {{< /file >}} - -- **Objects**: These are derived from classes and populate the abstract of their classes' properties with concrete values. They are the things built from the blueprints provided by classes. Objects also tend to be where the behaviors defined on classes get executed, bringing your application to life. - - Java lets you instantiate an object from a class using the `new` keyword. Here, a new object gets created from the class created above. This example works when the class has a *constructor* defined. You can see an example of a constructor definition in the [Examples of Object Oriented Programming](/cloud/guides/oop-principles#examples-of-object-oriented-programming) section further on. - - {{< file >}} -ClassName objectName = new ClassName(); - {{< /file >}} - -- **Attributes**: These are fields (or properties) defined on classes and which represent the state of a particular object. A class might, for instance, define an `attributeOne` as a `String` type. An object derived from that class can then use that attribute, assigning it `attributeOne = "a string"`, for example. - - This next example shows what it could look like to add an attribute to the `ClassName` class created above: - - {{< file >}} -public class ClassName { - public attributeOne = "a string"; -} - {{< /file >}} - -- **Methods**: These are functions defined on classes, and they provide objects with behaviors. Methods typically act on the values held by an object's attributes, allowing each object to act in a self-contained way. - - In the following example, you can see what it looks like to add a basic method to a class, using the `ClassName` example started above: - - {{< file >}} -public class ClassName { - public attributeOne = "a string"; - - public void methodOne() { - System.out.println("The method has been called!"); - } -} - {{< /file >}} - -## 4 Basic OOP Principles - -In addition to the four basic parts, object-oriented programming has four fundamental concepts. These are what primarily make OOP stand out, and developers rely on these when making the most effective and reusable OOP code. - -These next four sections cover the four principles of OOP, giving you an overview of the roles they play. Then, keep reading to find a section with examples, in Java, each of which demonstrates these principles in action. - -### Encapsulation - -This principle ensures that objects are self-contained and limits what information about their state they expose. In other words, other objects cannot directly access the state of an object. Each object manages its own state. To modify an object's state, other objects need to use that object's dedicated methods. - -So, for instance, say you have an object called `firstObject`. That object has two attributes, `attributeOne` and `attributeTwo`. Encapsulation prevents another object, say `secondObject`, from modifying the values of the attributes on `firstObject`. - -Now, `firstObject` has control of its own state. It may, for instance, define a method called `setAttributeOne` that outside objects can access. This way, `secondObject` can make changes to `attributeOne` on `firstObject`. But if `firstObject` does not define a similar method for `attributeTwo`, `secondObject` has no means of modifying it. - -Encapsulation can make OOP applications easier to upgrade and easier for collaboration. An engineer working on one object would thus be less likely to cause breaking changes to an object someone else is working on. - -Encapsulation also makes it easier to keep track of objects' states. These states can become complicated, and more so the more outside access they allow. By ensuring that each object controls its own state, you make the code easier for yourself and other developers to follow and maintain. - -### Data Abstraction - -This principle states that classes include only the details relevant to their context. Doing so creates abstract classes, which more specific classes and objects can extend. - -Take the example of a `Pet` class. You can make this class to define, in the most general way, the characteristics of pets. So, the class may have `name`, `diet`, and `health` attributes. Now you can extend that class with more specific kinds of `Pet`. For instance, you may define a `Dog` class that extends `Pet` and adds a `bark` method. At the same time, you can also define a `Cat` class similarly extending on `Pet`. - -One of the goals of abstraction is to define common characteristics. Using the example above, `Dog` has the unique behavior of the `bark` method, but otherwise it shares things like having a `name` in common with other pets. Abstraction makes it so that you do not need to redefine these attributes for each specific kind of pet. - -Abstraction also allows you to evaluate various classes by common abstract classes. So long as you know that both `Cat` and `Dog` extend `Pet`, you can evaluate them based on the common attributes held in `Pet`. - -{{< file >}} -if (obj eitherCatOrDog instanceof Pet) { - System.out.println("This is my pet, " + eitherCatOrDog.name + "."); -} -{{< /file >}} - -### Inheritance - -This principle declares that objects get some or all of the properties of their parents. Inheritance is the foundation of reusability in OOP. With it, you can create a class and its properties can be reused in multiple objects. - -For example, you can start with a `ClassName` class from which you create two objects, `objectOne` and `objectTwo`. Each of these objects inherits from the parent class, `ClassName`, and receives all of its attributes and methods. The objects can then each individually work with those attributes and methods. But the important feature is that the `ClassName` class acts as a common and reusable base. - -In Java, such parent classes are called *super* classes. Commonly, classes inheriting from super classes are called *sub* classes. This means that you can make additional classes that inherit from super classes, so that you can have a chain of inheritance. - -Take a look at the `Pet` example above again. You have a `Cat` class and a `Dog` class that inherit from `Pet`, thus gaining its attributes. From there, you can create specific objects that inherit from the new classes: - -{{< file >}} -Pet myDog = new Dog(); -{{< /file >}} - -The new object inherits not only properties of the `Dog` class — like the `bark` method — but also those on the `Pet` class, like the `name` attribute. - -### Polymorphism - -This principle states that each sub class can be used in the same way as its parent class or parent classes. At the same time, each sub class may keep its own, distinct form of attributes and methods initially defined in a super class. - -Polymorphism is one of the more complicated features of OOP, but it plays a useful role. To help you understand it, below is an example that reworks the `Pet` example elaborated in the sections above. - -Say, for instance, when creating the `Pet` class, you include a method called `makeSound`: - -{{< file >}} -class Pet { - public String name = "None"; - public String diet = "Herbivore"; - public boolean healthy = true; - - public void makeSound() { - System.out.println("This is my pet sound."); - } -} -{{< /file >}} - -Obviously, the effect of `makeSound` should be different for `Cat` and `Dog`, even though both, being pets, do make sounds: - -{{< file >}} -class Cat extends Pet { - public String diet = "Carnivore"; - - public void makeSound() { - System.out.println("Meow."); - } -} - -class Dog extends Pet { - public String diet = "Omnivore"; - - public void makeSound() { - System.out.println("Bark."); - } -} -{{< /file >}} - -Following polymorphism, you can, indeed, use any property from the `Pet` class on any object deriving from the `Cat` and `Dog` classes. The effect may be different — you get a different sound from the `makeSound` method — but all of the parts are still there. - -## Examples of Object Oriented Programming - -This section includes snippets of code that give examples of OOP concepts in Java. These are aimed to simultaneously show off some of the components of OOP as well as the four core principles discussed above. The examples also familiarize you with the elements of Java that relate to OOP. - -Starting simple, this first example shows a single Java class, not counting the default `Main` class used to start up the program. This class covers all of the parts — class, object, attribute, and method — of OOP mentioned above. - -{{< file >}} -// Create a class. -class BookShelf { - // Declare the class attributes. - public int numberOfBooks; - - // Implement a constructor. This is used to create objects from the class, - // which you can see done in the `Main` class below. - public BookShelf(int initialNumberOfBooks) { - numberOfBooks = initialNumberOfBooks; - } - - // Provide a method to add more books to the shelf. - public void addBooks(int numberToAdd) { - numberOfBooks += numberToAdd; - } - - // Provide a method to display the count of books on the shelf. - public void showBookCount() { - System.out.println("The shelf has " + numberOfBooks + " books."); - } -} - -class Main { - public static void main(String args[]) { - // Use the `BookShelf` class's constructor to create a BookShelf object; - // it also lets us specify how many books the object starts with. - BookShelf thisBookShelf = new BookShelf(5); - thisBookShelf.addBooks(2); - thisBookShelf.showBookCount(); - } -} -{{< /file >}} - -{{< output >}} -The shelf has 7 books. -{{< /output >}} - -Now, this next example is a little more ambitious. It has three classes — again, not counting the `Main` class. The first, `GamingConsole`, acts as a `super` on which other classes can extend. That is exactly what the second class, `PlayStation`, does — extends on the `GamingConsole` class. The last class, `PlayStation4`, does the same, but with the `PlayStation` class as its direct parent. - -This chain of extensions lets the example demonstrate several of the concepts of OOP at once. Each extension shows the concept of *abstraction* in action. The `PlayStation` class is able to make use of both attributes and methods from its parent, demonstrating *inheritance*. And the `PlayStation4` class illustrates *polymorphism* through its identification with the `GamingConsole` super class during construction in the `Main` class. - -{{< file >}} -// Create a super class, from which the other classes ultimately extend. -class GamingConsole { - // Declare attributes. - public String consoleType; - public String currentGame; - - // Provide a cosntructor to set initial values. - public GamingConsole(String initialConsoleType, String initialGame) { - consoleType = initialConsoleType; - currentGame = initialGame; - } - - // Provide two methods that should be common to all gaming consoles. - public void insertGame(String gameName) { - currentGame = gameName; - } - - public void playGame() { - System.out.println("Starting up the " + consoleType + " console."); - if (currentGame == "") { - System.out.println("No game in the console."); - } else { - System.out.println("Playing " + currentGame + "."); - } - } -} - -// Create a sub class for a specific category of gaming console. -class PlayStation extends GamingConsole { - // Declare attributes. - public boolean controllerConnected; - - // Provide a constructor. - public PlayStation(String initialGame, boolean initialControllerConnected) { - super("PlayStation", initialGame); - controllerConnected = initialControllerConnected; - } - - // Provide a method unique to this category of gaming consoles. (This - // feature is not actually unique to PlayStation consoles, but just - // pretend for the purposes of illustration.) - public void connectController(boolean isControllerConnected) { - controllerConnected = isControllerConnected; - } - - // Override the default `playGame` method with a specific implementation - // for `PlayStation` objects. - public void playGame() { - if (controllerConnected == false) { - System.out.println("Connect a controller before playing."); - } else { - super.playGame(); - } - } -} - -// Create another sub class for an even more specific category, this being -// a type of PlayStation gaming console. -class PlayStation4 extends PlayStation { - // Provide a constructor. - public PlayStation4(String initialGame, boolean initialControllerConnected) { - super(initialGame, initialControllerConnected); - consoleType = "PlayStation 4"; - - initiateWelcome(); - } - - // Provide a specific method for PlayStation 4 consoles. - public void initiateWelcome() { - System.out.println("Welcome to " + consoleType + "."); - } -} - - - -class Main { - public static void main(String args[]) { - // Instantiate a `PlayStation4` object. Notice the polymorphism - // implied by the fact that we can use `GamingConsole` to identify - // the new object's type. - GamingConsole thisConsole = new PlayStation4("", true); - - // Use the two methods inherited from the `GamingConsole` super class. - thisConsole.insertGame("Minecraft"); - thisConsole.playGame(); - } -} -{{< /file >}} - -{{< output >}} -Welcome to PlayStation 4. -Starting up the PlayStation 4 console. -Playing Minecraft. -{{< /output >}} - -## Conclusion - -In this guide you learned the fundamental principles of object-oriented programming. The concepts covered were encapsulation, abstraction, inheritance, and polymorphism. Applying these concepts helps to ensure that you are making the most of what the paradigm can do. - -Throughout this tutorial, the focus has been on OOP related to Java. But keep in mind that these concepts apply anywhere that supports object-oriented programming. [JavaScript](/cloud/guides/development/javascript), [Python](/cloud/guides/development/python), and [Ruby](/cloud/guides/development/ror) are popular examples. - - diff --git a/docs/guides/development/concepts/types-of-api/index.md b/docs/guides/development/concepts/types-of-api/index.md deleted file mode 100644 index dede50fe711..00000000000 --- a/docs/guides/development/concepts/types-of-api/index.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -slug: types-of-api -title: "What are the Four Types of API Formats?" -title_meta: "A Guide to API Formats: The Different Types of APIs" -description: "What are the different types of APIs and when should you use them? This guide discusses types of API and their different protocols." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2022-04-08 -keywords: ['types of apis','api formats','api types'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -APIs are what keep software connected. Whether you are looking to link your application to others or you want to have smooth communication between services, APIs help bring multiple pieces of an application together. - -Applications and services can be connected in myriad ways, depending on access limitations and communication protocols. APIs have developed several different approaches for making connections to support modern application architectures. - -In this tutorial, learn about what APIs are, the types of APIs that are available, and the various protocols they can use to communicate. - -## What is an API? - -An API — short for *Application Programming Interface* — defines a set of rules by which applications and services can interact. - -APIs are used in a wide variety of contexts. However, often, when people talk about APIs, they are talking about *web APIs*. These APIs allow for communication between applications and services using the [HTTP protocol](/cloud/guides/introducing-http-2). - -Often, web APIs are used for web application servers and web browsers to communicate. However, you may also see web APIs used for communication between different web servers, or between applications on the same server. You may even see web APIs at work between different services acting as parts of the same application. One example of an API enabling communication between different services of the same application, is Kubernetes. The [Kubernetes API](/cloud/guides/beginners-guide-to-kubernetes-part-1-introduction#kubernetes-api) is the linchpin to its powerful orchestration system. - -## The Four Main Types of APIs - -APIs come in four different modalities. Each of these covers a different access level or, in the case of web APIs, a different usage. - -Which one of these you use depends on your API's particular needs. The sections below provide descriptions of each kind of API and they can help you decide which is best for your use case. Each section also provides context and examples to make it easier to see how each API model can fit into different use cases. - -### Open APIs - -Open APIs, or public APIs, come with limited or no access restrictions. This essentially allows any developer to make requests to these APIs. - -These APIs may have some limits. A developer may have to register an account to receive an API key, for instance. Additionally, limits may be placed on things like the number of requests in a given time frame. - -But overall, open APIs are distinguished by being intended for widespread external use. They are meant for third-party developers to be able to access and make use of the API as they need. - -An example of open APIs are those provided by [NASA](https://api.nasa.gov/). After completing a simple registration for an API key, NASA gives you access to numerous open APIs. NASA's open APIs include everything from Earth observation images to information about the weather on Mars. - -#### When to Use an Open API? - -Make your API open when you intend it for public consumption. Open APIs are especially useful when you have information or services you want to make available to the general public. - -These APIs are often used for open source projects and for the dissemination of public knowledge, like NASA and other government agencies. - -### Partner APIs - -Partner APIs require authorization of some kind to use. They still allow external access, but are not intended for the general public to have access to. Instead, partner APIs are designed for use by pre-approved individuals, teams, or organizations. - -A partner API may allow public access through a paid subscription or it may limit access to developers with a business relationship. Typically, the developer has an API key, as with open APIs, that require registration. But with partner APIs, keys tend to be given out more sparingly and with more access restrictions. - -An example of a partner API is one that allows two companies to work together. Company A may have an application which Company B has agreed to provide services for. Developers at Company A receive API keys which they can use to access Company B's API. This allows Company A's application to make use of Company B's services while keeping access to these services limited. - -#### When to Use a Partner API? - -Make your API a partner API when it needs to be accessed externally but that access needs to be limited to authorized users. Partner APIs are ideal for business-to-business services or for subscription-based APIs. - -You are likely to see partner APIs in companies that make use of external services for parts of an application's functionality. Often, this can be a preferred solution compared to developing services in house. It allows companies to integrate features that have been developed by experts elsewhere into their applications. At the same time, it lets the external experts retain control of their services. - -### Internal APIs - -Internal APIs, also called private APIs, disallow external access. Instead, these APIs can only be accessed by developers within a company or even within the particular application to which the API belongs. - -These APIs are the most limited. APIs are incredibly useful in defining communication between applications and services, and this even applies when communication is within a single organization. - -A simple example of an internal API use case is a company that has two applications for selling items. One application allows customers to purchase items directly; the other allows sales personnel to process sales. Both applications need access to the inventory. The company could have both applications independently access the inventory database. However, doing so would likely lead to more difficult and inconsistent maintenance. - -So, instead, the company has an internal API for managing inventory. Both the customer-facing and sales-personnel applications can access this API to view and update inventory. Updates to each application can be made independently, as long as each adheres to the rules of the API. - -#### When to Use an Internal API? - -Make your API internal when you want to restrict access as much as possible. Internal APIs are designed to be private, with only applications and services within your organization having access. An internal API can even be used when different parts of an application need to communicate. - -These APIs are common within enterprise organizations. When applications scale, it helps to define APIs for managing underlying logic. Take the example above, where business logic can be developed and maintained in the customer-facing and sales-personnel applications. This can be done without concern for the impact to the underlying data storage and retrieval tasks, since those are housed in the internal API. - -### Composite APIs - -Composite APIs allow for requests to be bundled or chained together, which, in turn, allows developers to receive single responses for request collections. - -These APIs are useful for reducing server load and network traffic when you expect frequent requests to multiple API endpoints. Calls get made less frequently, resulting in reductions to server processing time and the number of requests across the network. - -This makes composite APIs exceptionally effective for microservices. Often, applications built on microservices have to compile information from multiple sources. Having composite APIs that do this makes for more efficient applications. - -To give an example of a composite API in action, think of an online ordering form. When the user completes and submits the form, the application often has to register the user, check and update inventory, and send a confirmation notification. A composite API allows all of these tasks to be handled simultaneously, in a single call. - -#### When to Use a Composite API? - -Make use of a composite API when your application exposes endpoints that are likely to be called in groups or in quick succession. This is often the case with microservices, where requests and responses frequently need to be combined. - -This type of API can be especially useful when your [microservice application](/cloud/guides/deploying-microservices-with-docker#what-is-a-microservice) needs to communicate with users' web browsers. Here, you want to optimize network traffic to reduce load times and improve user experience. You also want to reduce your server load to make your application scalable for a larger number of users. - -## What are the Different API Protocol Types? - -Every API uses a particular protocol. An API's protocol defines the rules for how it can communicate. These rules make explicit the kinds of requests that can be made, what the API's responses look like, and what kinds of data the API can send and receive. - -There are three main protocols used by web APIs. - -- **REST**. Short for Representational State Transfer, REST implements stateless APIs with uniform interfaces using HTTP. REST is actually more of a set of architectural principles for APIs than a protocol proper. You can use the [Flask Python framework](/cloud/guides/create-restful-api-using-python-and-flask) to build your own REST API. - -- **SOAP**. The Simple Object Access Protocol uses XML for requests and responses and maintains strict definitions for messages. SOAP is highly adaptable, designed to be neutral, and applicable in many contexts, not just for web APIs. It can even be used in conjunction with REST principles. - -- **RPC**. Simpler than both REST and SOAP, the Remote Procedural Call protocol focuses on actions taken on a server. This is in contrast to both REST and SOAP, which tend to focus on server resources. RPC works primarily on running processes. Often, RPC APIs execute scripts on the server. - -## Conclusion - -This guide has walked you through the basics of APIs, explaining the different categories they fit into and the contexts they are used in. The four main types of APIs are open, partner, internal, and composite. The guide also covered the protocols web APIs use to send and receive messages. These API protocols are REST, SOAP, and RPC. You now have a strong foundation for entering into the world of web APIs. It is a wide and fast-moving world. - diff --git a/docs/guides/development/frameworks/dotnet/install-dotnet-on-ubuntu/index.md b/docs/guides/development/frameworks/dotnet/install-dotnet-on-ubuntu/index.md deleted file mode 100644 index 817258af8c1..00000000000 --- a/docs/guides/development/frameworks/dotnet/install-dotnet-on-ubuntu/index.md +++ /dev/null @@ -1,154 +0,0 @@ ---- -slug: install-dotnet-on-ubuntu -title: "Install the .NET Runtime (or SDK) on Ubuntu 22.04" -title_meta: "How to Install the .NET Runtime (or SDK) on Ubuntu 22.04" -description: 'Learn how to install the .NET 6 SDK or runtime on Ubuntu and understand its limitations, requirements, and security implications.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2023-06-13 -keywords: ['dotnet on Ubuntu 22.04', 'dotnet runtime', 'install dotnet', 'enable dotnet ubuntu 22.04', 'install dotnet server ubuntu'] -tags: ['ubuntu'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -Microsoft [.NET](https://dotnet.microsoft.com/en-us/) (pronounced *dot net* and sometimes written as *dotnet*) is a free and open-source platform for building, distributing, and running software applications. Developers can write code for their applications in multiple languages (including C# and Visual Basic) and target any operating system that supports .NET (including Windows, Linux, and macOS). Using the .NET platform (and it's many available libraries and app models), developers can create command-line apps, web applications (with [ASP.NET](https://dotnet.microsoft.com/en-us/apps/aspnet)), cross-platform mobile and desktop applications (with [.NET MAUI](https://dotnet.microsoft.com/en-us/apps/maui)), and much more. - -.NET applications can be distributed as either *self-contained* or *framework-dependent* executables. Self-contained applications already include the necessary .NET runtime files and libraries needed to run the application. When running a *framework-dependent* application, you first need to install the .NET runtime and any required .NET libraries as these are not included. This guide covers how to install the .NET runtime (or SDK) so that you can run framework-dependent .NET applications on Ubuntu 22.04, provided these apps target the Linux operating system. - -## Platform Compatibility - -Applications built using .NET can support a wide range of both desktop and mobile operating systems, including Windows, Linux, macOS, iOS, and Android. This wasn't always the case, which is why some resources online might describe .NET as Windows-only. Originally, .NET existed as the .NET Framework. This was a proprietary software development platform built by Microsoft to support the development of Windows applications. The last version of the .NET Framework (version 4.8) was released in 2019. - -In 2016, Microsoft released .NET Core (what is now simply called .NET) as a completely new implementation of the platform. It was developed specifically for cross-platform support and was released as a free and open-source software development platform. It has now replaced the .NET Framework and, starting with the release of .NET "Core" version 5, the word "core" was dropped from the name. - -## The .NET SDK vs the .NET runtime - -When running a framework-dependent .NET application on Linux, you need to first install .NET. On most distributions, there are typically several packages available, including those for the .NET SDK and the .NET runtime. The SDK (short for software development kit) contains tools needed to develop *and* run applications. The runtime just contains the files needed to run .NET applications. If you do not intend to develop .NET applications on your system, you can save space by installing the runtime instead of the SDK. - -## Installation - -Microsoft's .NET runtime does not come pre-installed on most Linux distributions, including Ubuntu 22.04 LTS. Unless you have already installed Visual Studio or another application that's dependent on the same .NET runtime version, you'll need to install the runtime before using your .NET app. - -The .NET runtime can be installed directly from Ubuntu's package repositories (starting with Ubuntu 22.04), from Microsoft's own repository, or by using the Snap package manager. - -### Determine if .NET is Already Installed - -To avoid conflicting or duplicate .NET installations, you should check if any .NET runtimes are already installed on your system. To do this, run the command below: - -```command -dotnet --list-runtimes -``` - -If .NET is not installed, you will likely receive a command not found message. If .NET is installed, your output should list each runtime that is installed. - -```output -Microsoft.AspNetCore.App 7.0.5 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App] -Microsoft.NETCore.App 7.0.5 [/usr/lib/dotnet/shared/Microsoft.NETCore.App] -``` - -For a more comprehensive view of .NET on your system, use the `dotnet --info` command. - -### Install .NET Using APT - -Starting with Ubuntu 22.04 LTS, .NET is available from Ubuntu's own package repositories and can quickly be installed using the APT package manager. For more details on this installation procedure, review Microsoft's [Install .NET SDK or .NET Runtime on Ubuntu 22.04](https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu-2204) guide. - -{{< note >}} -Not all .NET versions are included in Ubuntu's repository. If your application requires a specific .NET version (including nightly versions), you should use Microsoft's own repository instead. For instructions on doing this, review the official [Register the Microsoft package repository](https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#register-the-microsoft-package-repository) guide. Using mixed repositories can potentially lead to conflicts, update issues, and asset version conflicts between the repositories. Microsoft provides [detailed information on the current ramifications](https://github.com/dotnet/core/issues/7699) and suggests solutions on resolving repository problems. - -{{< /note >}} - -The following .NET packages are available when using the default package repositories on Ubuntu 22.04: - -- [dotnet7](https://packages.ubuntu.com/jammy-updates/dotnet7): The main .NET 7.0 package, which includes CLI tools, the SDK, and the runtime. -- [dotnet6](https://packages.ubuntu.com/jammy-updates/dotnet6): The main .NET 6.0 package, which includes CLI tools, the SDK, and the runtime. - -You can also install specific components of either version using the packages below: - -- [dotnet-sdk-7.0](https://packages.ubuntu.com/jammy-updates/dotnet-sdk-6.0): The .NET 7.0 SDK. This includes both the .NET runtime and ASP.NET runtime, among other tools and packages. -- [dotnet-runtime-7.0](https://packages.ubuntu.com/jammy-updates/dotnet-runtime-6.0): The .NET 7.0 runtime. -- [aspnetcore-runtime-7.0](https://packages.ubuntu.com/jammy-updates/aspnetcore-runtime-6.0): The ASP.NET 7.0 runtime, which also installs the .NET 7.0 runtime. -- [dotnet-sdk-6.0](https://packages.ubuntu.com/jammy-updates/dotnet-sdk-6.0): The .NET 6.0 SDK. This includes both the .NET runtime and ASP.NET runtime, among other tools and packages. -- [dotnet-runtime-6.0](https://packages.ubuntu.com/jammy-updates/dotnet-runtime-6.0): The .NET 6.0 runtime. -- [aspnet-runtime-6.0](https://packages.ubuntu.com/jammy-updates/aspnetcore-runtime-6.0): The ASP.NET 6.0 runtime, which also installs the .NET 6.0 runtime. - -You can choose to install either the main package, the SDK, the .NET runtime, or the ASP.NET runtime (for ASP.NET web applications). If you only intend to run an existing .NET application and wish to conserve disk space (or avoid installing unnecessary software), we recommend just installing the runtime package corresponding with whichever .NET version you want to use. - -To install .NET using APT, open a terminal and run the following command to update local sources and install the desired package: - -```command -sudo apt update && sudo apt install [package] -``` - -Replace *[package]* with the name of the .NET package you want to install (such as `dotnet7` or `dotnet6`). - -### Install .NET Using Snap - -*Snap* is a package manager that is pre-installed all newer Ubuntu systems, including Ubuntu 22.04 LTS. Snap packages are installed as an instance for the user invoking snap and these instances are not shared with other users by default. Ubuntu’s snap packages use the `squashfs` file system, which enables the package contents to be placed into the current user's directory. The snap package installation process is automatic, and updates are handled seamlessly. - -It's worth noting that while snap is the default packaging method for Ubuntu, Microsoft's Visual Studio IDE for Linux does not currently have a snap package available. Therefore, if you are using Microsoft's Visual Studio, you need to install the .NET SDK separately. - -{{< note title="Check for incorrectly installed or shared snap packages." isCollapsible=true type="secondary" >}} -To check for the presence of a shared instance of .NET (SDK or runtime) that is incorrectly installed for sharing, you can follow these steps: - -1. Execute the following command to test for the shared instance: - - ```command - sudo ls /home/user/share/dotnet - ``` - - This command checks if the directory where .NET shared code is typically stored exists. The `/home/user/share/dotnet` directory is commonly used for storing the shared code. Replace `user` with the actual username of the user account you are using. - -1. If the shared instance is found, it needs to be removed or isolated from the desired user. This can be achieved by either: - - - Removing the shared instance by identifying the directories or files associated with the shared .NET code. This could include the `/home/user/share/dotnet` directory mentioned earlier, as well as any other directories or files related to the shared instance. Use appropriate commands such as `rm` (remove) or `sudo rm` (remove with root privileges) to delete the identified directories or files. - - - Installing a snap-only instance to separate it from the desired user. The specific command may vary depending on the name of the dotnet snap package. For example, if the package is named `dotnet-sdk`, you can use the following command: - - ```command - sudo snap install dotnet-sdk - ``` -{{< /note >}} - -To install .NET using Snap, follow the steps below: - -- **.NET SDK:** Install the full .NET SDK package by running the following command: - - ```command - sudo snap install dotnet-sdk --classic --channel=6.0 - ``` - - This command installs the .NET 6.0 SDK. If you wish to install .NET 7.0 SDK, use `--channel=7.0`. - -- **.NET runtime:** To install the .NET runtime, run the following command: - - ```command - sudo snap install dotnet-runtime-60 - ``` - - This installs the .NET 6.0 runtime. Other versions are also available, such as for .NET 7.0 (`dotnet-runtime-70`) and 3.1 (`dotnet-runtime-31`). - -For a full list of the available snap packages, run the command below: - -```command -snap find dotnet -``` - -This queries the snap repository and displays a list of available .NET SDK and runtime versions. Note that versions marked as "preview" or "RC" (release candidate) are not considered stable. - -## Server Applications - -While many .NET applications are standalone, others connect directly to external systems. If your application connects to other services, you may need to adjust your firewall settings so these connections are not blocked. For instructions on using UFW (the default firewall front-end interface for Ubuntu 22.04), see the guide [How to Configure a Firewall with UFW](/cloud/guides/configure-firewall-with-ufw). Web services typically use ports `443`, `80`, and `8080`, though ports are application-specific. If these ports are blocked, you can adjust the firewall to allow access. - -Scripts installing .NET applications may require sudo rights to effectively change the firewall, files, or environmental settings for the user(s) of the runtime application. In addition, you may need to modify user rights and file accessibility so that your .NET-based application can properly run. - -## Limitations of .NET in Ubuntu 22.04 - -The .NET components in the Ubuntu Repositories (apt and snap) support certain versions of .NET, but not all versions. If an application relies on a specific version that is not available, you may need to perform an alternative installation method. Consult the application's documentation (or contact its developer) to learn what operating systems are supported and what specific version dependencies are needed to run the application. - -## Security Implications - -Since Ubuntu 22.04 includes support for .NET 6.0 and .NET 7.0, many security risks present in older versions of .NET do not apply. However, older versions of .NET (like version 3.1 and 5.0) can be installed using Snap or other methods. When older versions of .NET are used, consider any security implications before running the software. While stand-alone code that does not access data from external sources is unlikely to present a security risk, applications that do connect to external services should be scrutinized. Starting with .NET version 6.0, applications benefit from the enhanced security provided by OpenSSL version 3. Previous versions rely on the now-deprecated encryption methods of OpenSSL version 1, which are known to have security vulnerabilities. - -## Conclusion - -Now that Ubuntu (Canonical) and Microsoft offer .NET packages through Ubuntu's default repositories, installing .NET runtimes (or SDKs) is much easier on Ubuntu 22.04 (and later versions). Whether you're installing .NET from APT, Snap, or another method, you can now install and run .NET applications quickly, and cleanly, and with the security that comes with consistent package updates. While many older versions of the .NET runtime versions are not supported through APT, the more secure .NET 6.0 and 7.0 versions can be installed on any Ubuntu system. \ No newline at end of file diff --git a/docs/guides/development/graphql/graphql-apollo-an-introduction/index.md b/docs/guides/development/graphql/graphql-apollo-an-introduction/index.md deleted file mode 100644 index 8c998eb3609..00000000000 --- a/docs/guides/development/graphql/graphql-apollo-an-introduction/index.md +++ /dev/null @@ -1,242 +0,0 @@ ---- -slug: graphql-apollo-an-introduction -title: "GraphQL Apollo: An Introduction with Examples" -title_meta: "An Introduction to GraphQL Apollo" -description: 'In this guide, you will learn what GraphQL is and how to use the Apollo GraphQL client to manage local and remote data from your command line in Linux.' -authors: ["Cameron Laird"] -contributors: ["Cameron Laird"] -published: 2021-09-03 -keywords: ['what is apollo graphql', 'apollo graphql client', 'apollo graphql server'] -tags: ['web applications'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[GraphQL](https://graphql.org/)' -- '[Apollo](https://www.apollographql.com/)' -- '[Apollo GraphQL documentation](https://www.apollographql.com/docs/)' ---- - -GraphQL is a server-side runtime and query language for APIs. Known for its speed, GraphQL is ideal for limited-bandwidth edge devices, and applications that benefit from GraphQL's bandwidth-saving query format. Compared to representational state transfer (REST), GraphQL lets you retrieve all the data you need in a single query. GraphQL began as an internal project at Facebook in 2012. After a public release in 2015, the [GraphQL Foundation](https://graphql.org/foundation/) was created in 2018 to provide a permanent home for GraphQL. - -The example below displays a basic GraphQL query. The query retrieves data from specific fields on the example `realtor` object: - -{{< output >}} -{ - realtor { - name - } -} -{{< /output >}} - -The result of the query returns the following JSON output: - -{{< output >}} -{ - "data": { - "realtor": { - "name": "Terry Musa" - } - } -} -{{< /output >}} - -## What is Apollo GraphQL? - -As a public specification, there are several GraphQL implementations. [Apollo](https://www.apollographql.com/) maintains the most prominent of these implementations. Apollo GraphQL is the collection of implementations the Apollo company bases on the GraphQL standard. Apollo’s implementation is largely [open-source](https://www.apollographql.com/docs/apollo-server/). In addition to its open-source products, Apollo offers specialized paid services that bring value to large-scale applications of GraphQL. - -## What is the Apollo GraphQL Client? - -In the GraphQL model, data is sent between a *client* and *server*. The client issues a request to the server, and the server answers the client’s request. The Apollo client is a JavaScript state management library that helps you fetch data and manipulate the data using concise syntax and development best practices. It is possible to use the Apollo GraphQL client with any GraphQL server; the latter doesn't have to be from Apollo. - -## Apollo GraphQL vs REST - -[Representational state transfer](https://searchapparchitecture.techtarget.com/definition/REST-REpresentational-State-Transfer) (REST) is still widely used and has [advantages over GraphQL](https://blog.logrocket.com/why-you-shouldnt-use-graphql/#whyandwhentousegraphql). - -Some of the advantages of REST include: - -- Community maturity. REST has been around since 2000. There are many resources supporting its usage and many experienced developers who are able to work with REST. -- Data format support. REST supports many types of data formats including CSV, HTML, JSON, and XML. -- Decoupled client and server implementations. When using REST, a client does not require any specialized libraries to query the server data. - -Some of the advantages of GraphQL include: - -- Prevents over fetching of data. You can retrieve all the data you need in a single query. -- Schema versioning is not required. Instead of versioning, GraphQL utilizes a schema registry to track an API's evolution. -- An extensive ecosystem of libraries. There are many libraries available to extend GraphQL's behavior. - -To become more familiar with GraphQL's capabilities refer to the [Apollo blog's post on the benefits of GraphQL](https://www.apollographql.com/blog/graphql/basics/why-use-graphql/). - -### Apollo GraphQL Client Example - -The example in this section queries an open [GraphQL service](https://api.spacex.land/graphql/) that [SpaceX](https://www.spacex.com/) provides. Before beginning the steps in this section, ensure you have [installed Node.js using the Node Version Manager](/cloud/guides/how-to-install-nodejs-and-nginx-on-ubuntu-18-04#install-nodejs). - -From your system's command line, install the GraphQL client: - - npm install graphql @apollo/client cross-fetch -{{< note >}} -Your system may require [ECMAScript module (esm) loader](https://www.npmjs.com/package/esm). To install esm, use the following command: - - sudo npm install --save esm -{{< /note >}} - -The installation should take a few seconds to complete. - -Create a new file named `client-example.js` with the following content: - -{{< file "client-example.js" >}} -// cross-fetch helps ensure correct behavior independent of the environment -// needs the HttpLink. -import fetch from 'cross-fetch'; -const fetch = require('cross-fetch'); -import { - ApolloClient, gql, HttpLink, InMemoryCache -} from "@apollo/client"; - -const cache = new InMemoryCache() -const uri = 'https://api.spacex.land/graphql/'; -const query = gql` - query { - launchesPast(limit: 10) { - mission_name - } - }` -const link = new HttpLink({uri: uri, fetch}); -const client = new ApolloClient({link: link, cache: cache}); - -client - .query({query: query}) - .then(result => console.log(JSON.stringify(result))); -{{< /file >}} - -This code imports the necessary libraries to then query the SpaceX API. The query expects a JSON response from the API. - -Run the `client-example.js` file with the following command: - - node -r esm client-example.js - -You should see a similar JSON output returned: - -{{< output >}} -{"data":{"launchesPast":[{"__typename":"Launch","mission_name":"Starlink-15 (v1.0)"},{"__typename":"Launch","mission_name":"Sentinel-6 Michael Freilich"},{"__typename":"Launch","mission_name":"Crew-1"},{"__typename":"Launch","mission_name":"GPS III SV04 (Sacagawea)"},{"__typename":"Launch","mission_name":"Starlink-14 (v1.0)"},{"__typename":"Launch","mission_name":"Starlink-13 (v1.0)"},{"__typename":"Launch","mission_name":"Starlink-12 (v1.0)"},{"__typename":"Launch","mission_name":"Starlink-11 (v1.0)"},{"__typename":"Launch","mission_name":"SAOCOM 1B, GNOMES-1, Tyvak-0172"},{"__typename":"Launch","mission_name":"Starlink-10 (v1.0) & SkySat 19-21"}]},"loading":false,"networkStatus":7} -{{< /output >}} - -Using your preferred JavaScript framework, you can update the code to request more data from the API and to render the data in a browser. - -## Apollo GraphQL Server - -Several implementations of a [GraphQL server](https://blog.graphqleditor.com/graphql-servers) are available to use. Apollo is a good choice given its robust documentation and potential for longevity. Several well-known companies use Apollo for their projects, including Airbnb, the New York Times, and CircleCI. - -### Server Installation Steps - -Before beginning the steps in this section, ensure you have [installed Node.js using the Node Version Manager](/cloud/guides/how-to-install-nodejs-and-nginx-on-ubuntu-18-04#install-nodejs) on your server. - -To install the Apollo GraphQL server use the following command: - - npm install apollo-server-express express graphql - -Create a new filled named `hello-server.js` and add the following content: - -{{< file "hello-server.js" >}} -var {graphql, buildSchema} = require('graphql'); - -var schema = buildSchema(` - type Query { - hello: String - } -`); - -var root = {hello: () => 'Hello, world!'}; - -graphql(schema, '{ hello }', root) - .then((response) => { - console.log(response); -}); -{{< /file >}} - -Run the above example file using the following command - - node hello-server.js - -You should see a similar output: - -{{< output >}} -{ data: { hello: 'Hello, world!' } } -{{< /output >}} - -The example demonstrates a GraphQL server response. The next step in server behavior is to embed this responsiveness into a web API. To do this, update the `server-example.js` with the following content: - -{{< file "server-example.js" >}} -const express = require('express'); -const {ApolloServer, gql} = require('apollo-server-express'); - -const typeDefs = gql` - type Query { - hello: String - }`; -const resolvers = { - Query: { - hello: () => 'Hello, world!', - }, -}; - -async function startApolloServer() { - const app = express(); - const server = new ApolloServer({ typeDefs, resolvers }); - const message = 'Please browse to ' + server.graphqlPath; - - //You must "await server.start()" before calling "server.applyMiddleware()" - //"await" can only be used inside an async() fun - await server.start(); - server.applyMiddleware({ app }); - - app.listen({ port: 4003 }, () => - console.log(message) - ); -} - -startApolloServer(); - -{{< /file >}} - -The code above defines the GraphQL schema, creates a resolver, and creates an instance of the Apollo server that is served on the localhost. - -Run the code in the example file with the following command: - - node server-example.js - -The output returns the following: - -{{< output >}} -Please browse to -{{< /output >}} -{{< note >}} -To visit the application remotely, you can use an SSH tunnel: - -- On macOS or Linux, use the command below to set up the SSH tunnel. Replace `example-user` with your username on the application server and `192.0.2.0` with the server's IP address. - - ssh -L 4000:localhost:4000 example-user@192.0.2.0 - -Now you can visit the application in your browser by navigating to `https://localhost:4000`. -{{< /note >}} - -If you visit `http://localhost:4000/graphql` in a browser, you see an empty instance of a playground Integrated Development Environment (IDE). Enter the following query in the left window of the playground IDE. - - query {hello} - -Push the central play button with the right-pointing arrow icon. You can see the following output pop up on the right. - -{{< output >}} -{ - "data": { - "hello": "Hello, world!" - } -} -{{< /output >}} - -This section's example provides a simple model for creating a GraphQL service. - -## Conclusion - -Apollo GraphQL is a powerful platform that can take some time investment to learn to use. You can leverage free web tools to experiment and to model your GraphQL data. For example, use the [GraphQL IDE](https://github.com/graphql/graphiql) and the [Sandbox Explorer](https://studio.apollographql.com/sandbox/explorer) to test your ideas. The GraphQL Foundation maintains a page on [best practices](https://graphql.org/learn/best-practices/) that you can visit to learn more about the specification. - - - diff --git a/docs/guides/development/java/how-to-install-openjdk-on-centos-8/index.md b/docs/guides/development/java/how-to-install-openjdk-on-centos-8/index.md deleted file mode 100644 index d16140199f4..00000000000 --- a/docs/guides/development/java/how-to-install-openjdk-on-centos-8/index.md +++ /dev/null @@ -1,146 +0,0 @@ ---- -slug: how-to-install-openjdk-on-centos-8 -title: Installing Java Development Kit 11 on CentOS 8 -title_meta: How To Install Java Development Kit 11 on CentOS 8 -description: 'This guide will show you how to install the free and open-source Open Java Development Kit (OpenJDK) version of the Java Runtime Environment (JRE) on CentOS 8.' -authors: ["Rajakavitha Kodhandapani"] -contributors: ["Rajakavitha Kodhandapani"] -published: 2020-03-25 -keywords: ["java", "openjdk", "jdk", "11", "CentOS", "8"] -tags: ["java","centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Installing_Java_Development_Kit_11_on_CentOS8_1200x631.png -audiences: ["beginner"] -languages: ["java"] -relations: - platform: - key: install-openjdk - keywords: - - distribution: Ubuntu 16.04 -aliases: [] ---- - -[Java](https://www.oracle.com/java/index.html) is one of the world's most popular programming languages. Software written in Java can be compiled and run on any system, making Java a versatile platform that can be used to create anything from software to basic web applications. This guide will show you how to install the Open Java Development Kit (OpenJDK) 11 on CentOS 8. - -OpenJDK is the free and open-source implementation of the Oracle Java Standard Edition (Java SE) Development Kit. OpenJDK and Java SE are equivalent JDKs that include a Java runtime environment (JRE) and tools for developing and compiling Java applications. - -While there are many available versions of OpenJDK, version 11 is the latest Long-Term-Support (LTS) release as of the time of this guide's publication. For this reason, OpenJDK 11 is the recommended version for developing production applications. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for connecting to your Linode with SSH and setting your Linode's hostname and timezone. - -1. Complete the sections of our guide on [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) to create a standard user account, harden SSH access and remove unnecessary network services. This guide will use `sudo` commands wherever possible, which should be run by a limited, non-root user on your Linode. - -1. Ensure your system is up-to-date: - - sudo yum update - -## Install OpenJDK - -1. Install the OpenJDK 11 development kit, which includes OpenJRE 11: - - sudo yum install java-11-openjdk-devel - - Alternatively, if you simply want to run Java applications that you have already downloaded, you can choose to only install OpenJRE 11: - - sudo yum install java-11-openjdk - - {{< note respectIndent=false >}} -While you can run Java applications directly with the JRE, your applications will be compiled every time they are executed. This is generally slower than running applications that have already been compiled into Java bytecode, and may not be suitable if you plan to execute applications many times. -{{< /note >}} - -1. Check the version of the JRE to verify that it has been properly installed: - - java -version - - As of the time of this publication, this command should return: - - {{< output >}} -openjdk version "11.0.7" 2020-04-14 LTS -OpenJDK Runtime Environment 18.9 (build 11.0.7+10-LTS) -OpenJDK 64-Bit Server VM 18.9 (build 11.0.7+10-LTS, mixed mode, sharing) -{{< /output >}} - -1. If you have chosen to install the full OpenJDK development kit, check the version of the compiler as well: - - javac -version - - As of the time of this publication, this command should return: - - {{< output >}} -javac 11.0.7 -{{< /output >}} - -## Set Environment Variables - -This section will instruct you on how to set the `JAVA_HOME` and `PATH` environment variables to help ensure that your Java applications will run without issue. - -1. Open the `~/.bashrc` startup file using the text editor of your choice and add the following definitions at the end of the file: - - {{< file "~/.bashrc" sh>}} -# [...] -export JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))) -export PATH=$PATH:$JAVA_HOME/bin -{{< /file >}} - - {{< note respectIndent=false >}} -If you are using a shell other than Bash, such as [Zsh](https://github.com/ohmyzsh/ohmyzsh), you may need to add these lines in a different startup file instead. In the case of Zsh, this would be the `~/.zshrc` file. -{{< /note >}} - -1. Save the changes and exit your text editor. - -1. Reload the `~/.bashrc` file: - - source ~/.bashrc - -1. Verify that the JAVA_HOME and PATH variables were set correctly: - - echo $JAVA_HOME - echo $PATH - - The `JAVA_HOME` variable should be set to the directory that contains your OpenJDK installation, and the `PATH` variable should include the directory that contains the OpenJDK binary files. - -## Test the Java Installation (Optional) - -To test your Java installation, write a sample `HelloWorld` Java application and run it with the JRE. - -1. Open a text editor and add the following lines in a file labeled `HelloWorld.java` to create a simple function that prints "Hello Java World!": - - {{< file "HelloWorld.java" java >}} -public class HelloWorld { - public static void main(String[] args) { - System.out.println("Hello Java World!"); - } -} -{{< /file >}} - -1. Run the application using the JRE: - - java HelloWorld.java - - If the installation has been successful, the output will be: - - {{< output >}} -Hello Java World! -{{< /output >}} - -If you have installed the full OpenJDK development kit, you can compile your application into a bytecode class file prior to running it for faster execution time. - -1. Compile the application you have written: - - javac HelloWorld.java - -1. Confirm that the `HelloWorld.class` file was written to your current directory: - - ls -l HelloWorld.class - -1. Run the compiled `HelloWorld` function using the JRE: - - java HelloWorld - - The output should again be: - - {{< output >}} -Hello Java World! -{{< /output >}} diff --git a/docs/guides/development/java/how-to-install-openjdk-on-ubuntu-18-04/index.md b/docs/guides/development/java/how-to-install-openjdk-on-ubuntu-18-04/index.md deleted file mode 100644 index 0c4bf5e750f..00000000000 --- a/docs/guides/development/java/how-to-install-openjdk-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,147 +0,0 @@ ---- -slug: how-to-install-openjdk-on-ubuntu-18-04 -title: "Installing OpenJDK 11 (Java Development Kit) on Ubuntu 18.04" -title_meta: "Install OpenJDK 11 (Java Development Kit) on Ubuntu 18.04" -description: 'This guide will show you how to install the Open Java Development Kit (OpenJDK) 11 on Ubuntu 18.04 & test it. ✓ Click here to read our guide today!' -og_description: 'This guide will show you how to install the Open Java Development Kit (OpenJDK) 11 on Ubuntu 18.04 and test it by through a simple "Hello World" application.' -authors: ["Rajakavitha Kodhandapani"] -contributors: ["Rajakavitha Kodhandapani"] -published: 2020-03-25 -keywords: ["java", "openjdk", "jdk", "11", "Ubuntu", "18.04"] -aliases: [] -tags: ["java","ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Installing_Java_Development_Kit_11_on_Ubuntu1804_1200x631.png -audiences: ["beginner"] -languages: ["java"] -relations: - platform: - key: install-openjdk - keywords: - - distribution: Ubuntu 18.04 ---- - -[Java](https://www.oracle.com/java/index.html) is one of the world's most popular programming languages. Software written in Java can be compiled and run on any system, making Java a versatile platform that can be used to create anything from software to basic web applications. This guide will show you how to install the Open Java Development Kit (OpenJDK) 11 on Ubuntu 18.04. - -OpenJDK is the free and open-source implementation of the Oracle Java Standard Edition (Java SE) Development Kit. OpenJDK and Java SE are equivalent JDKs that include a Java runtime environment (JRE) and tools for developing and compiling Java applications. - -While there are many available versions of OpenJDK, version 11 is the latest Long-Term-Support (LTS) release as of the time of this guide's publication. For this reason, OpenJDK 11 is the recommended version for developing production applications. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for connecting to your Linode with SSH and setting your Linode's hostname and timezone. - -1. Complete the sections of our guide on [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) to create a standard user account, harden SSH access and remove unnecessary network services. This guide will use `sudo` commands wherever possible, which should be run by a limited, non-root user on your Linode. - -1. Ensure your system is up-to-date: - - sudo apt-get update && sudo apt-get upgrade - -## Install OpenJDK - -1. Install the OpenJDK 11 development kit, which includes OpenJRE 11: - - sudo apt-get install openjdk-11-jdk - - Alternatively, if you simply want to run Java applications that you have already downloaded, you can choose to only install OpenJRE 11: - - sudo apt-get install openjdk-11-jre - - {{< note respectIndent=false >}} -While you can run Java applications directly with the JRE, your applications will be compiled every time they are executed. This is generally slower than running applications that have already been compiled into Java bytecode, and may not be suitable if you plan to execute applications many times. -{{< /note >}} - -1. Check the version of the JRE to verify that it has been properly installed: - - java -version - - As of the time of this publication, this command should return: - - {{< output >}} -openjdk version "11.0.7" 2020-04-14 LTS -OpenJDK Runtime Environment 18.9 (build 11.0.7+10-LTS) -OpenJDK 64-Bit Server VM 18.9 (build 11.0.7+10-LTS, mixed mode, sharing) -{{< /output >}} - -1. If you have chosen to install the full OpenJDK development kit, check the version of the compiler as well: - - javac -version - - As of the time of this publication, this command should return: - - {{< output >}} -javac 11.0.7 -{{< /output >}} - -## Set Environment Variables - -This section will instruct you on how to set the `JAVA_HOME` and `PATH` environment variables to help ensure that your Java applications will run without issue. - -1. Open the `~/.bashrc` startup file using the text editor of your choice and add the following definitions at the end of the file: - - {{< file "~/.bashrc" sh>}} -# [...] -export JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))) -export PATH=$PATH:$JAVA_HOME/bin -{{< /file >}} - - {{< note respectIndent=false >}} -If you are using a shell other than Bash, such as [Zsh](https://github.com/ohmyzsh/ohmyzsh), you may need to add these lines in a different startup file instead. In the case of Zsh, this would be the `~/.zshrc` file. -{{< /note >}} - -1. Save the changes and exit your text editor. - -1. Reload the `~/.bashrc` file: - - source ~/.bashrc - -1. Verify that the JAVA_HOME and PATH variables were set correctly: - - echo $JAVA_HOME - echo $PATH - - The `JAVA_HOME` variable should be set to the directory that contains your OpenJDK installation, and the `PATH` variable should include the directory that contains the OpenJDK binary files. - -## Test the Java Installation (Optional) - -To test your Java installation, write a sample `HelloWorld` Java application and run it with the JRE. - -1. Open a text editor and add the following lines in a file labeled `HelloWorld.java` to create a simple function that prints "Hello Java World!": - - {{< file "HelloWorld.java" java >}} -public class HelloWorld { - public static void main(String[] args) { - System.out.println("Hello Java World!"); - } -} -{{< /file >}} - -1. Run the application using the JRE: - - java HelloWorld.java - - If the installation has been successful, the output will be: - - {{< output >}} -Hello Java World! -{{< /output >}} - -If you have installed the full OpenJDK development kit, you can compile your application into a bytecode class file prior to running it for faster execution time. - -1. Compile the application you have written: - - javac HelloWorld.java - -1. Confirm that the `HelloWorld.class` file was written to your current directory: - - ls -l HelloWorld.class - -1. Run the compiled `HelloWorld` function using the JRE: - - java HelloWorld - - The output should again be: - - {{< output >}} -Hello Java World! -{{< /output >}} diff --git a/docs/guides/development/javascript/getting-started-with-svelte/index.md b/docs/guides/development/javascript/getting-started-with-svelte/index.md deleted file mode 100644 index 48143cb0656..00000000000 --- a/docs/guides/development/javascript/getting-started-with-svelte/index.md +++ /dev/null @@ -1,237 +0,0 @@ ---- -slug: getting-started-with-svelte -title: "Getting Started with the Svelte Framework" -description: "The Svelte framework handles frontend development by shifting work to the build step, giving you small and fast application bundles without a virtual DOM. Learn all about Svelte, what sets it apart, and how to start using it in this guide." -authors: ['Nathaniel Stickman'] -contributors: ['Nathaniel Stickman'] -published: 2022-05-02 -keywords: ['svelte guide', 'svelte javascript', 'svelte vs react'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Svelte: Documentation](https://svelte.dev/docs)' -- '[SvelteKit Docs: Introduction](https://kit.svelte.dev/docs/introduction)' -- '[MDN Web Docs: Getting Started with Svelte](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Client-side_JavaScript_frameworks/Svelte_getting_started)' -- '[Twilio Blog: Beginner’s Guide to Svelte - From Basics to State Management and Animation](https://www.twilio.com/blog/all-you-need-to-know-svelte)' ---- - -Svelte is a JavaScript framework for frontend web development. And unlike most similar frameworks, Svelte handles page processing at build time. Svelte frontends operate without a virtual DOM, giving you small and efficient application bundles with streamlined performance. - -Through this guide, learn more about what sets the Svelte framework apart and how you can start building your own Svelte frontends. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -{{< note >}} -The steps in this guide are written for non-root users. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Linux Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## What Is the Svelte Framework? - -[Svelte](https://svelte.dev/) is a JavaScript framework for developing web application frontends. There are many such frameworks, and these typically offload the majority of work to a user's browser through a virtual Document Object Model (DOM). Svelte sets itself apart by shifting the bulk of that work to the app compilation step. Svelte frontends, as a result, work without a virtual DOM. - -Svelte's leveraging of the compilation step means that its application bundles tend to be smaller and more performant. Couple that with Svelte's more classical frontend model, and you have a compelling tool for efficiency and approachability. - -### Svelte vs React vs Vue - -React remains one of the most widely used frameworks for web-frontend development, and the popularity of the Vue framework is not too far behind. Relative to these frameworks, Svelte is a newcomer. What, then, makes Svelte different, and why would you choose it over React, or Vue? - -Both React and Vue handle web page changes through a virtual DOM. The virtual DOM provides efficiency over traditional DOM manipulation, but it still has the user's browser handle most of the page processing. As a result, users can experience performance dips with more virtual DOM manipulation. - -Svelte diverges from that model. Instead of using a virtual DOM, Svelte applications do the majority of their process during compilation. The result is small, performant application bundles. - -Svelte also stays closer to standard HTML and JavaScript/TypeScript concepts and tooling. React and Vue re-imagine the model for web-frontend development, introducing many novel concepts and tools to enhance the developer experience. However, Svelte's model only slightly deviates from classic frontend models. That fact makes the framework typically more approachable for developers unfamiliar with modern web frontend concepts and tooling. - -## How to Install Svelte - -To install Svelte, the recommended approach in the official documentation is to start your Svelte project using the SvelteKit. The SvelteKit installs with the command for creating a new NPM project, meaning you do not need any global Svelte installation. - -With the SvelteKit installed, your project compiles `.svelte` files into the appropriate JavaScript and CSS at build time. SvelteKit also brings ready access to convenient frontend development features like routing. - -1. Install NPM to manage your Svelte project and install its dependencies. Follow the relevant section of our guide on [How to Install and Use the Node Package Manager (NPM) on Linux](/cloud/guides/install-and-use-npm-on-linux#how-to-install-npm). - -1. Create a new NPM project using the SvelteKit template. This example names the project `exampleApp`, and this name gets used for the Svelte project throughout the rest of this guide: - - ```command - npm create svelte@latest exampleApp - ``` - - Follow the prompts to complete the setup for your project. - - - Choose an application template to start your project with. This guide later uses the `Skeleton project` for a base template, but you may choose the `SvelteKit demo app` option to seem Svelte's capabilities. - - - Choose whether you would like type-checking within your project. This is where you can select TypeScript for your project should you want. Svelte also has the option of type-checking through a `JSDoc` syntax, which is what this guide uses. Choose `No` for a more traditional JavaScript experience, without type checking. - - - Choose any additional features you would like from the remaining prompts. This guide selects `No` to all of these subsequent prompts to make the examples simpler, but the features offered here can improve the development experience for many projects. - -1. Change into the new project directory, and have NPM install the project dependencies. The rest of this guide assumes you are in this directory for any example commands: - - ```command - cd exampleApp - npm install - ``` - -And with that, you have gotten started with Svelte. Anytime you want to create a new Svelte application, you can use the steps above to set it up. - -You can test this base project by running the development server. Run the development server using the following command, and then look for the output by navigating to `localhost:5173` in a web browser: - -```command -npm run dev -``` - -```output - VITE v4.0.1 ready in 1165 ms - - ➜ Local: http://localhost:5173/ - ➜ Network: use --host to expose - ➜ press h to show help -``` - -To access this URL from a remote machine, you can use an SSH tunnel. Set up the SSH tunnel using one of the methods below, depending on your operating system. - -- On **Windows**, you can use the PuTTY tool to set up your SSH tunnel. Follow the PuTTY section of our guide on how to [Create an SSH Tunnel for MySQL Remote Access](/cloud/guides/create-an-ssh-tunnel-for-mysql-remote-access#how-to-access-mysql-remotely-by-creating-an-ssh-tunnel-with-putty). Use `5173` as the **Source port** and `127.0.0.1:5173` as the **Destination**. - -- On **macOS** or **Linux**, use the following command to set up the SSH tunnel. Replace `example-user` with your username on the remote server and `192.0.2.0` with the remote server's IP address: - - ```output - ssh -L5173:localhost:5173 example-user@192.0.2.0 - ``` - -You should now be able to access the development server from the remote machine. Navigate to `localhost:5173` in a web browser, just as you would on the remote machine itself. - -![The welcome page for a default Svelte project](svelte-default.png) - -## How to Build a Frontend with Svelte - -With a Svelte project of your own created, you should now get familiar with how Svelte applications work. Svelte adheres more closely to traditional web development models, making it often more approachable for new frontend developers. However, SvelteKit does implement some particular structures to help modern projects off the ground. - -These next sections aim to get you familiar with the SvelteKit structure and its features as well as the fundamentals of developing a Svelte application. The example application demonstrates some of the characteristics of Svelte applications overall. - -### SvelteKit Application Structure - -Using the SvelteKit base template, your project gets initialized with a simple structure that you can build on. Here is what the base structure looks like: - -- `src/` - - - `routes/` - - - `+page.svelte` - -- `app.d.ts` - -- `app.html` - -Most of the work for building your application takes place within the `src/routes` path. Within this directory, any `+page.svelte` files represent the base page template for the path. So the `+page.svelte` file above, in the `routes` directory, represents the landing page, for the `/` address. - -Adding a `new-path` directory under `routes` creates a new route. A `+page.svelte` file within this new directory would define a page for the `/new-path` path. - -There are numerous more features for working with the `routes` structure. What follows are a few of the most noteworthy and useful for getting started. - -- Use square brackets (`[...]`) in a `routes` path to implement a slug. For instance, a `routes/new-path/[slug]` directory creates a new path with a variable slug. JavaScript and Svelte files within the `[slug]` directory can access that slug as a parameter. - -- Use a `+page.js` file (or `+page.ts` for TypeScript) within a `routes` path to implement a script for loading data. These scripts get executed before the path's page gets rendered. The scripts can be used, for instance, to fetch page content based on a parameter like a path slug. - -- Use `+error.svelete` within a `routes` path to create custom error pages. SvelteKit has default error pages, but using `+error.svelte` lets you provide customized displays and responses. - -Learn more about the SvelteKit structure and the features it offers through the SvelteKit documentation linked at the end of this guide. - -### Example Svelte Application - -The base template set up with the installation above gives only a sparse demonstration of Svelte, without showing routing and interactive features. So to help better understand Svelte and the SvelteKit features, the following series of steps walk you through creating a basic interactive frontend. - -The example creates a to-do list, adding a new route and a link to it from the main page. The new route includes a script to load the initial to-do list and demonstrates features like dynamic rendering and two-way data binding. - -1. Follow the section above on installing Svelte to create a new Svelte project. Use the `Skeleton project` template, and choose the `JSDoc` option for the type checking. - -1. Open the `src/routes/+page.svelte` file, and modify its contents to include the following. This just changes the paragraph text and includes a link to a `/todo` route: - - ```file {title="src/routes/+page.svelte"} -

Welcome to Your Svelte App!

-

Visit your to-do list to manage your tasks.

- ``` - -1. Create a new directory of `src/routes/todo`. This creates a new `/todo` route for your application. - -1. Add a `+page.js` file within the `src/routes/todo` directory, and give the file the contents shown here. The `load` function runs before the page renders, usually to load data. The example here simply uses some static data, but the function is capable of fetching data from other files or services: - - ```file {title="src/routes/todo/+page.js"} - /** @type {import('./$types').PageLoad} */ - export function load({ params }) { - return { - todoList: ['Test 1', 'Test 2', 'Test 3'], - } - } - ``` - -1. Add a `+page.svelte` file within the `src/routes/todo` directory, and give the file the contents shown here. Annotations are provided throughout the code here to help break down what each part does. Overall, this file defines the structure of the to-do list and the functions for handling actions on the list: - - ```file {title="src/routes/todo/+page.svelte"} - - -

Svelte To-do List

- -
-

Add a to-do item

-
- - - -
-
- -
- - {#each todoList as todoItem} -

{todoItem}

- {:else} -

Nothing to do!

- {/each} -
- ``` - -1. Run the development server for the application, and navigate to it in your web browser, just as shown in the section on installing the project above. You should see a variant of the default welcome page. - - ![The modified welcome page for the example Svelte application](svelte-example-welcome.png) - -1. Click the link on the welcome page or otherwise navigate to the `/todo` route, `localhost:5173/todo`, to see the default to-do list. - - ![A basic to-do list in the example Svelte application](svelte-example-todo-1.png) - -1. Make some changes to the list to verify that it works as expected. Here, the item `Test 2` was removed and items `Test 4` and `Test 5` were added. - - ![The example Svelte to-do list applications with dynamic modifications to the displayed list](svelte-example-todo-2.png) - -## Conclusion - -You now have a working Svelte application of your own and with it a foundational knowledge of how to work with Svelte and SvelteKit. Svelte has many more features for you to leverage, and SvelteKit still helps you structure modern and efficient frontend applications. Be sure to refer to the Svelte and SvelteKit documentation linked below to get the most out of your new Svelte application. diff --git a/docs/guides/development/javascript/how-to-write-vue-js-plugins/index.md b/docs/guides/development/javascript/how-to-write-vue-js-plugins/index.md deleted file mode 100644 index cbecb0eb420..00000000000 --- a/docs/guides/development/javascript/how-to-write-vue-js-plugins/index.md +++ /dev/null @@ -1,393 +0,0 @@ ---- -slug: how-to-write-vue-js-plugins -title: "Write Your Own Vue.js Plugin" -description: 'Learn how to create your own Vue.js plugin with an easy-to-use package and add functionality to your Vue apps.' -authors: ["John Au-Yeung"] -contributors: ["John Au-Yeung"] -published: 2021-02-09 -keywords: ['vue','vue.js','plugin','directive', 'filter', 'mixin'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[VueJS](https://vuejs.org/)' -tags: ["web applications"] ---- - -## What is a Vue.js Plugin? - -A Vue.js plugin is a code package that is used to add features to a Vue.js app. The plugin is structured with JavaScript and exported with an `install` method that takes the `Vue` object as the first parameter and `options` as the second parameter. - -The `Vue` parameter is the Vue instance of the current Vue app. With it, you can add your own methods, mixins, and directives with the following standards: - -* To add component instance methods, which is available to all components, attach a new property to the `Vue.prototype` property with your desired function. - -* The method has to be a traditional function because it references the Vue instance with `this`. - -* To add directives, call the `Vue.directive` method with the directive name string and an object with the directive lifecycle hooks as the arguments. - -* Likewise, use the `Vue.mixin` method to declare your own mixin objects. - -* You can also declare a global `Vue` method by attaching a new property to it with a function as its property. - -To use the plugin in the app, then call the global `Vue.use` method with the plugin as the first argument and an optional object with options that the plugin takes as the second argument. - -The options object in the second argument is available as the value of the `options` parameter in the `install` function. - -## Creating a Simple Vue.js Plugin -{{< note >}} -The code below assumes that the Vue project is created by the [Vue CLI](https://cli.vuejs.org/). -{{< /note >}} - -{{< file "src/plugin.js" js >}} -export default { - // eslint-disable-next-line no-unused-vars - install(Vue, options) {} -}; -{{< /file >}} - -{{< file "src/main.js" js >}} -import Vue from "vue"; -import App from "./App.vue"; -import Plugin from "./plugin"; - -Vue.use(Plugin); - -Vue.config.productionTip = false; - -new Vue({ - render: h => h(App) -}).$mount("#app"); -{{< /file >}} - -The code above, the `plugin.js` file is a plugin that has nothing except the empty `install` function in the object that is exported as the default export: - -The plugin is registered in the `main.js` file by calling the `Vue.use` method with the `Plugin` object imported from `plugin.js` as the argument. Then, everything in the plugin becomes available in all parts of the app. - -With this skeleton code created, you can add something simple like a method that you can use in the component. For instance, you can add a method that can be used in the component by writing the following code: - -{{< file "src/plugin.js" js >}} -export default { - // eslint-disable-next-line no-unused-vars - install(Vue, options) { - Vue.prototype.$toBold = function(text) { - return `${text}`; - }; - } -}; -{{< /file >}} - -{{< file "src/App.vue" html >}} - - - -{{< /file >}} - -In the code above, you added a new method to the `Vue.prototype` property, which is the `$toBold` method. We added the `$` sign in front of the method name to distinguish it from the methods that is created in components. - -Then use the `$toBold` method in the component by calling it with a string of your choice to display the string as bold. - -`main.js` remains the same as in the previous example. - -You can define global properties by attaching a property to the `Vue` parameter and setting a value to it. - -For example, write the following: - -{{< file "src/plugin.js" js >}} -export default { - // eslint-disable-next-line no-unused-vars - install(Vue, options) { - Vue.PLUGIN_VERSION = "0.0.1"; - } -}; -{{< /file >}} - -{{< file "src/App.vue" html >}} - - - -{{< /file >}} - -In the code above, you attached the `PLUGIN_VERSION` property as a property of the `Vue` object. - -Then, import `Vue` in `App.vue` to get the value of the `Vue.PLUGIN_VERSION` property as you did in `App.vue`'s code. - -In the template, call the `getVersion` method that is defined to show the value of the `PLUGIN_VERSION` property. - -In the browser, '0.0.1' appears. - -## Adding Directives to a Vue.js Plugin - -You can add directives by calling the `Vue.directive` method. It takes a string for the directive name and an object with the lifecycle hooks for a directive. - -A Vue directive has a few hooks. They include the `bind` hook which is called once when the directive is first bound to the component to run initial setup code. - -The `inserted` hook is called when the bound element has been inserted into its parent node. When this hook is run, the parent node is present but not necessarily in the document itself. - -Finally, the `update` hook is called after the containing component’s VNode has updated, which the DOM node in the virtual DOM of Vue has updated. This hook might be run before the child nodes have updated. You can get the old and new value that is set as the value of the directive in the `update` hook. - -To add a directive into the plugin that can be used in the components throughout the app, write the following code: - -{{< file "src/plugin.js" js >}} -export default { - // eslint-disable-next-line no-unused-vars - install(Vue, options) { - Vue.directive("highlight", { - inserted(el) { - el.style.color = "green"; - } - }); - } -}; - -{{< /file >}} - -{{< file "src/App.vue" html >}} - - - -{{< /file >}} - -In the code above, a new directive called `highlight` is defined in `plugins.js`. - -Then use it in `App.vue` by adding `v-highlight` in the component. The highlighted text "foo" displays in green color because you set the content of the element `el` (the `p` element in `App.vue`) to green. - -Next, pass the values into the directive and use it as follows: - -{{< file "src/plugin.js" js >}} -export default { - // eslint-disable-next-line no-unused-vars - install(Vue, options) { - Vue.directive("highlight", { - inserted(el, { value }) { - el.style.color = value ? value : "green"; - } - }); - } -}; -{{< /file >}} - -{{< file "src/App.vue" html >}} - - - -{{< /file >}} - -The code above gets the `value` property from the `binding` parameter in the 2nd position. The `value` property's value is passed in after the equals sign in the template. Therefore, `value` is set to `blue`, so the text displays as blue rather than the default green. - -## Adding Mixins to a Vue.js Plugin - -You can add mixins to add code that can merged in with a existing code of the component if it is used. - -When you declare a mixin inside a plugin, then the methods in the mixin object are merged into all components. - -You can create a mixin by calling the `Vue.mixin` method as follows: - -{{< file "src/plugin.js" js >}} -export default { - // eslint-disable-next-line no-unused-vars - install(Vue, options) { - Vue.mixin({ - async beforeMount() { - const res = await fetch("https://api.agify.io/?name=example"); - const { name } = await res.json(); - this.name = name; - } - }); - } -}; -{{< /file >}} - -{{< file "src/components/Foo.vue" html >}} - - - -{{< /file >}} - -{{< file "src/App.vue" html >}} - - - -{{< /file >}} - -In the code above, you defined the global mixin with `Vue.mixin` method. Then inside the mixin object, the `beforeMount` hook runs code to get data from an API before all components are mounted. - -Then in `App.vue` and `Foo.vue`, you display the value of the `name` state without adding any code to the component thanks to the mixin. - -The `beforeMount` hook runs automatically whenever any component is mounted. Therefore, "example" is displayed twice because it fetches the name in the API in each component with the `beforeMount` hook in the mixin. - -You should be careful with global mixin code because it runs in all components. - -## Adding Filters to a Vue.js Plugin - -You can add a global filter to the plugin by calling the `Vue.filter` method. The method takes the name string as the first argument and a function that takes the value that you want to transform and return something that is in the format that you want as the return value. - -For example, define a filter in the plugin as follows: - -{{< file "src/plugin.js" js >}} -export default { - // eslint-disable-next-line no-unused-vars - install(Vue, options) { - Vue.filter("timeString", val => { - if (!(val instanceof Date)) { - return val; - } - return val.toLocaleTimeString(); - }); - } -}; -{{< /file >}} - -{{< file "src/App.vue" html >}} - - - -{{< /file >}} - -In the code above, you defined a filter called `timeString` with the function returning a time string if it is the value passed in is an instance of the `Date` object. - -In `App`'s template, you applied the `timeString` filter on `new Date()`, which is the date object with the current date and time, with the pipe operator. - -Then, something similar to '5:02:49 PM' appears on the screen. - -## Accepting Vue.js Plugin Options - -You can pass in options to the plugin. To do that, you just have to pass in an object with options in the `Vue.use` as the 2nd argument. - -Then fetch it from the 2nd parameter of the `options` parameter. - -For example, pass in options and use it in the plugin as follows: - -{{< file "src/plugin.js" js >}} -export default { - install(Vue, { getDateString }) { - Vue.filter("dateOrTimeString", val => { - if (!(val instanceof Date)) { - return val; - } - - if (getDateString) { - return val.toDateString(); - } - return val.toLocaleTimeString(); - }); - } -}; -{{< /file >}} - -{{< file "src/main.js" js >}} -import Vue from "vue"; -import App from "./App.vue"; -import Plugin from "./plugin"; - -Vue.use(Plugin, { getDateString: true }); - -Vue.config.productionTip = false; - -new Vue({ - render: h => h(App) -}).$mount("#app"); -{{< /file >}} - -{{< file "src/App.vue" html >}} - - - -{{< /file >}} - -In the code above, you passed in `{ getDateString: true }` as the option object. - -In `plugin.js`, you destructured the `getDateString` property from the options object and used it in the filter. - -Then when you set the `getDateString` to `true`, a date string is displayed on the screen. Otherwise, the time string is displayed. - -## Conclusion - -You can define a plugin by creating a JavaScript module that exports an object with the `install` method as a default export. - -The `install` method has the `Vue` parameter, which has the Vue instance as its value, and `options` which has the options object as the 2nd argument. - -Then inside the plugin, you can add directives, methods, and mixins of our choice. diff --git a/docs/guides/development/nodejs/install-configure-selenium-grid-ubuntu-16-04/index.md b/docs/guides/development/nodejs/install-configure-selenium-grid-ubuntu-16-04/index.md deleted file mode 100644 index 08e4c7c4ab9..00000000000 --- a/docs/guides/development/nodejs/install-configure-selenium-grid-ubuntu-16-04/index.md +++ /dev/null @@ -1,260 +0,0 @@ ---- -slug: install-configure-selenium-grid-ubuntu-16-04 -title: 'Use Selenium Grid for Cross-Browser Compatibility Testing' -title_meta: 'Install and Configure Selenium Grid on Ubuntu 16.04 for Automated Testing' -description: 'This guide shows how to set up a Selenium grid that can be used to test webpages and applications in different browsers, browser versions, and operating systems.' -authors: ["Jared Kobos"] -contributors: ["Jared Kobos"] -published: 2018-03-09 -keywords: ["selenium", "node.js", " headless browser", "automation", "webdriver"] -tags: ["automation"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Selenium Project Home](https://www.seleniumhq.org/projects/webdriver/)' - - '[Selenium Node.js Documentation](http://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/index.html)' -audiences: ["intermediate"] -relations: - platform: - key: install-selenium-grid - keywords: - - distribution: Ubuntu 16.04 -aliases: [] ---- - -## What is Selenium Grid? - -[Selenium](https://seleniumhq.github.io/selenium/docs/api/javascript/index.html) is a browser automation library with bindings for most common programming languages. It is most often used for testing web applications, but can also be used to automate any task that a web browser can perform. In contrast to similar tools such as Nightmare.js, Selenium can run tasks or tests on any version of any major browser. This can make it more complicated to get running, but allows you to test your application's behavior on exactly the platforms that users are likely to need. - -For many applications, the Selenium standalone server is sufficient. However, Selenium can also be configured as a grid, with multiple nodes communicating with a central hub. The nodes and hub can be run on the same computer or server, or can be located on separate Linodes. Selenium Grid offers two main benefits: - -- Selenium runs on Java and is compatible with all major operating systems. This makes it possible to set up a grid consisting of servers (or virtual machines) running Linux, OSX, and Windows. A test suite can then be run against the grid, with each test run on multiple browsers and operating systems. The hub will delegate each test to a node that has the requested capability (e.g. sending a Safari test to a node with OSX and the Safari webdriver). - -- For larger projects, running a lengthy test suite in series can be time consuming. By running the test suite across a grid consisting of multiple servers, it is possible to distribute the tests across multiple nodes and significantly increase the performance of the testing process. - -This guide shows how to set up a simple Selenium grid consisting of a hub and two nodes, all on separate Linodes. A simple test script will then be used to demonstrate running tests against different versions of Firefox. - -## Prepare Grid Linodes - -Install Java and other dependencies on each Linode that will be part of the Selenium grid. This guide uses three Linodes for this purpose, but you can also run all of the nodes from the same Linode if you prefer. Throughout this guide, these Linodes will be referred to as `hub`, `node-1`, and `node-2`. - -### Install Java - -{{% content "install-java-8-ppa" %}} - -### Install Dependencies - -When running tests with Selenium, each grid node can only run tests on browsers that have been installed on that node. Each browser also requires a separate executable webdriver. For this example, install Geckodriver and different versions of Firefox on `node-1` and `node-2`. - -1. Check the latest release of Geckodriver on the [releases](https://github.com/mozilla/geckodriver/releases) page and download it to `node-1` and `node-2`: - - wget https://github.com/mozilla/geckodriver/releases/download/v0.19.1/geckodriver-v0.19.1-linux64.tar.gz - -2. Extract the archive and move the executable to a location in your PATH: - - tar -xvf geckodriver-v0.19.1-linux64.tar.gz - sudo mv geckodriver /usr/local/bin/ - -3. On `node-1`, install the latest stable version of Firefox: - - sudo apt install firefox - -4. On `node-2`, install the Firefox developer edition: - - sudo add-apt-repository ppa:ubuntu-mozilla-daily/firefox-aurora - sudo apt update && sudo apt install firefox - -5. Check and record the version numbers to use when running tests: - - **node-1:** - - firefox --version - - {{< output >}} -Mozilla Firefox 58.0.2 -{{< /output >}} - - **node-2:** - - firefox --version - - {{< output >}} -Mozilla Firefox 59.0 -{{< /output >}} - -### Download Selenium - -Selenium provides a single `.jar` file that can be used to run a standalone server, hub, or node. Check the latest release at the [Selenium downloads page](https://www.seleniumhq.org/download/) and download the file to each Linode using `wget`: - - wget http://selenium-release.storage.googleapis.com/3.10/selenium-server-standalone-3.10.0.jar - -### Start Grid Hub - -1. On the `hub` Linode, start the hub by running the Selenium server with `-role` set to `hub`: - - java -jar selenium-server-standalone-3.10.0.jar -role hub - -2. The resulting output will give you URLs to use for registering nodes and connecting to the hub. Copy these URLs for later use: - - {{< output >}} -21:27:51.470 INFO [GridLauncherV3.launch] - Selenium build info: version: '3.10.0', revision: '176b4a9' -21:27:51.475 INFO [GridLauncherV3$2.launch] - Launching Selenium Grid hub on port 4444 -2018-03-06 21:27:52.248:INFO::main: Logging initialized @1166ms to org.seleniumhq.jetty9.util.log.StdErrLog -21:27:52.446 INFO [Hub.start] - Selenium Grid hub is up and running -21:27:52.447 INFO [Hub.start] - Nodes should register to http://69.164.211.42:4444/grid/register/ -21:27:52.448 INFO [Hub.start] - Clients should connect to http://69.164.211.42:4444/wd/hub -{{< /output >}} - -### Configure Grid Nodes - -1. On `node-1` and `node-2`, create a node configuration file `config.json` and add the following content. Replace the `hub` address with the public IP of the `hub` Linode, and replace the `version` with the version of Firefox installed on the respective nodes. If you are putting the grid and nodes on the same Linode, replace the IP address with `http://localhost:4444`: - - {{< file "config.json" json >}} -{ - "capabilities": - [ - { - "browserName": "firefox", - "marionette": true, - "maxInstances": 5, - "seleniumProtocol": "WebDriver", - "version": "58.0.2" - } - ], - "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy", - "maxSession": 5, - "port": 5555, - "register": true, - "registerCycle": 5000, - "hub": "http://192.0.2.0:4444", - "nodeStatusCheckTimeout": 5000, - "nodePolling": 5000, - "role": "node", - "unregisterIfStillDownAfter": 60000, - "downPollingLimit": 2, - "debug": false, - "servlets" : [], - "withoutServlets": [], - "custom": {} -} -{{< /file >}} - -2. Connect each node to the hub: - - java -jar selenium-server-standalone-3.10.0.jar -role node -nodeConfig config.json - -3. You should see output similar to the following, indicating that your nodes have been successfully registered to the hub: - - {{< output >}} -21:33:22.856 INFO - Selenium Server is up and running on port 5555 -21:33:22.857 INFO - Selenium Grid node is up and ready to register to the hub -21:33:22.895 INFO - Starting auto registration thread. Will try to register every 5000 ms. -21:33:22.896 INFO - Registering the node to the hub: http://69.164.211.42:4444/grid/register -21:33:23.064 INFO - Updating the node configuration from the hub -21:33:23.178 INFO - The node is registered to the hub and ready to use -{{< /output >}} - - You can also check the output from the hub itself: - - {{< output >}} -21:27:53.849 INFO [DefaultGridRegistry.add] - Registered a node http://198.58.122.154:5555 -21:27:56.445 WARN [BaseRemoteProxy.] - Max instance not specified. Using default = 1 instance -21:27:56.450 INFO [DefaultGridRegistry.add] - Registered a node http://50.116.22.93:5555 -21:27:56.743 WARN [BaseRemoteProxy.] - Max instance not specified. Using default = 1 instance -{{< /output >}} - -4. Navigate to `http://192.0.2.0:4444/grid/console` in a web browser (replace `192.0.2.0` with the public IP address of your `hub` Linode) to see a console listing your available nodes. - - ![Selenium Grid Console](grid-console.png "Selenium Grid Console") - - The console should show that each node is configured to use a different version of Firefox. - -## Prepare Local Test Environment - -In this example, the test script will be run from your local development machine. It will connect to the remote grid and execute the tests from there. If you do not have an available development machine, use a separate Linode. - -### Install Node.js and NPM - -This guide uses the NPM `selenium-webdriver` package, which contains Node.js bindings for Selenium. - -{{% content "install-nodejs-nodesource" %}} - -### Create an Example Test Script - -This script tests the Linode Docs home page. - -1. Create a directory for the test suite: - - mkdir test-selenium && cd test-selenium - -2. Initialize a Node.js app within the directory: - - npm init - - Accept the default values when prompted. - -3. Install NPM packages: - - npm install --save selenium-webdriver - -4. Create `test.js` and add the following script. Replace `192.0.2.0` on Line 11 with the IP address of `hub`: - - {{< file "~/test-selenium/test.js" js >}} -const {Builder, By, Capabilities, Key, until} = require('selenium-webdriver'); -let firefox = require('selenium-webdriver/firefox'); - -const VERSIONS = ['58.0.2','59.0']; - -function buildDrivers(versions) { - let drivers = []; - for (let version of versions) { - driver = new Builder().forBrowser('firefox') - .withCapabilities(Capabilities.firefox().setBrowserVersion(version)) - .usingServer('http://192.0.2.0:4444/wd/hub') - .setFirefoxOptions( - new firefox.Options().headless()) - .build(); - drivers.push(driver); - } - console.log('built drivers for ' + versions); - return drivers; -} - -async function example(driver) { - try { - await driver.get('http://www.linode.com/docs'); - await driver.findElement(By.name('q')).sendKeys('nginx', Key.RETURN); - let el = driver.findElement(By.linkText('How to Configure nginx')); - await driver.wait(until.elementIsVisible(el), 1000); - await el.click(); - title = await driver.getTitle(); - } finally { - await driver.quit(); - } -}; - -async function main() { - const drivers = await buildDrivers(VERSIONS); - - for (let driver of drivers) { - example(driver); - }; -} - -main(); -{{< /file >}} - -5. Save the test script, then run it: - - node test.js - - If successful, the script will search for NGINX in the Linode docs library, visit one of the results pages, and check that the page title matches the link text. It will print out the page title as well: - - {{< output >}} -built drivers for 58.0.2,59.0 -How to Configure nginx -How to Configure nginx -{{< /output >}} - -6. The driver requested two versions of Firefox to run on the two nodes. `test.js` runs the driver asynchronously across two nodes in order to run the `getTitle` method across two browser versions. - -By creating different drivers for each combination of platform, browser, and version you want to test, you can specify which node or nodes should be used to run each test. If more than one node has the requested capabilities, Selenium will choose one of the nodes at random. In this way it is possible to run a large, cross-browser test suite in much less time than it would take to run the tests one at a time. diff --git a/docs/guides/development/nodejs/nodejs-twitter-bot/index.md b/docs/guides/development/nodejs/nodejs-twitter-bot/index.md deleted file mode 100644 index 5314f1c270d..00000000000 --- a/docs/guides/development/nodejs/nodejs-twitter-bot/index.md +++ /dev/null @@ -1,1092 +0,0 @@ ---- -slug: nodejs-twitter-bot -title: "Make a Twitter Bot and Reply to Tweets in Node.js" -description: "How to make a Twitter bot with Node.js, publish it to GitHub, and then host it on a Linode cloud instance. The guide is intended for people new to coding." -authors: ["Pj Metz"] -contributors: ["Pj Metz"] -published: 2021-07-23 -keywords: ["how to make a twitter bot", "node twitter", "reply bot twitter", "node twitter api tutorial"] -tags: ["version control system", "javascript"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - "[Documentation Home | Docs | Twitter Developer Platform](https://developer.twitter.com/en/docs)" - - "[npm Docs](https://docs.npmjs.com/)" - - "[node - npm](https://www.npmjs.com/package/node)" - - "[node-schedule - npm](https://www.npmjs.com/package/node-schedule)" - - "[dotenv - npm](https://www.npmjs.com/package/dotenv)" - - "[twit - npm](https://www.npmjs.com/package/twit)" - - "[GitHub Desktop](https://desktop.github.com/)" - - "[Creating a new User in Ubuntu](https://youtu.be/fDHHKR0nVQg)" - - "[FileZilla](/cloud/guides/filezilla)" ---- - -## Introduction - -Twitter bots are a great way to practice using an API and develop some coding skills along the way. You can usually get the bot going for under 100 lines of code, so it’s especially good for beginners. This tutorial shows how to make a Twitter bot that tweets a link to a video game soundtrack once a week (pulled randomly from an array of YouTube links). It also responds to other Twitter users who ask for a link with a random link from the same array. - -The tutorial demonstrates how to use Node.js, a framework for JavaScript, and a few packages from *npm*, the Node Package Manager, to have your bot tweeting in no time. As well, the guide shows how to set up a GitHub repository for your bot's code. At the end of the guide, we demonstrate how to host the bot on a Linode cloud instance. - -## Before You Begin - -### Create Twitter, GitHub, and Linode Accounts - -To follow along with this guide, accounts with the following websites are required: - -- [Twitter](https://twitter.com/home): If you already have a Twitter account, you can use that existing account to access the [Twitter Developer portal](https://developer.twitter.com/en). You may instead want to create a new, separate Twitter account to use for this tutorial. If you sign up for a new account, Twitter requires a unique email address and phone number to register the account. - - This is where you decide what you want your account to be named and what it does automatically. The tutorial names the bot [@SNESSoundtracks](https://twitter.com/snessoundtracks), but your Twitter account name needs to be unique. - -- [GitHub](https://github.com/): This tutorial shows how to record the bot's application code in a repository on GitHub. If you already have a GitHub account, you can use that one. If you do not, create a new one now. - -- [Linode](https://www.linode.com): At the end of the tutorial, the Node.js app is installed and run on a Linode instance. If you already have a Linode account, you can use that one. If you do not, create a new one now. For new accounts, there is a **Sign Up** link at the bottom of this tutorial that offers a free credit. - -### Install Software on your Workstation - -The following software is needed on your workstation to complete the tutorial: - -- **A plain-text editor**. [Visual Studio Code](https://code.visualstudio.com/) (abbreviated *VS Code*) is a recommended desktop text editor, but any other plain-text editor can be used as well. VS Code also includes a [built-in terminal](https://code.visualstudio.com/docs/editor/integrated-terminal). - -- **Git**: The Git command line software, or [GitHub Desktop](https://desktop.github.com/). - - These tools are used to download a copy of the GitHub repository that is created in this guide. The GitHub Desktop software can be more user-friendly for beginners, but you might prefer to work out of the command line. Instructions for using both are provided in this tutorial. - - To install the command line software, follow our [How to Install Git on Linux, Mac or Windows](/cloud/guides/how-to-install-git-on-linux-mac-and-windows) guide. Then, follow the [Configure Git](/cloud/guides/how-to-configure-git#configure-git) section of our [Getting Started with Git](/cloud/guides/how-to-configure-git) guide. When doing this, you don't need to set the `core.editor` option, but it is important to set your username and email for Git. - -- **[Node.js](https://nodejs.org/en/) and the [Node Package Manager](https://www.npmjs.com/) (*npm*)**, which are used to initialize a new npm package for the Twitter bot. - - There are a number of different ways to install Node.js and npm. Our [How to Install Node.js](/cloud/guides/how-to-install-nodejs) guide outlines some of these options. For this tutorial, we recommend using the *Node Version Manager* (*nvm*), which can manage multiple versions of Node.js and npm. Follow the next [Install Node.js and npm via the Node Version Manager](#install-nodejs-and-npm-via-the-node-version-manager) section for instructions. - -### Install Node.js and npm via the Node Version Manager - -The original [nvm](https://github.com/nvm-sh/nvm) software project is used on **Unix, Linux, and macOS**: - -1. Follow the [Install NVM](/cloud/guides/how-to-install-use-node-version-manager-nvm#install-nvm) section of our [How to Install and Use the Node Version Manager](/cloud/guides/how-to-install-use-node-version-manager-nvm) guide to install nvm. - -1. Run this command from your terminal. This installs the latest version of Node.js and npm: - - nvm install --lts - -A separate [nvm-windows](https://github.com/coreybutler/nvm-windows) project was created for native installation on **Windows**: - -1. Follow the [instructions in the project's README](https://github.com/coreybutler/nvm-windows#installation--upgrades) to install nvm-windows. - -1. Run this command from your terminal. This installs the latest version of Node.js and npm: - - nvm install latest - -## Apply for Access to the Twitter Developer Portal - -Twitter's developer portal is where you register your new account to be able to use the Twitter API. This is an important step because it unlocks the account's ability to tweet by using the API with code instead of manually tweeting. You know, the whole point of a bot? - -Follow these steps to register for developer access: - -1. Head to the [developer page](https://developer.twitter.com/en). Once there you’ll see a link near the profile picture on the top right that says **Apply**. - -1. Click the **Apply for a developer account** button on the page that appears: - - ![Twitter Developer Portal - Apply for access button](twitter-apply-for-access-button.png "Twitter Developer Portal - Apply for access button") - -1. If you are not already signed in, Twitter prompts you to sign in to your account. - -1. Twitter first asks what your intended use for the developer account is. For this tutorial, we have chosen the **Hobbyist** option, followed by the **Making a bot** option. - - ![Twitter developer application - which best describes you question with Hobbyist and Making a bot options selected](twitter-developer-application-which-best-describes-you.png "Twitter developer application - which best describes you question with Hobbyist and Making a bot options selected") - -1. Complete the remaining questions in the **Basic info**, **Indended use**, **Review**, and **Terms** screens. The **Indended use** screen asks you for more detail about your application. For this tutorial, these are the answers that were entered for each prompt: - - - **In your words**: Let Twitter know that you’re applying for developer access as a hobby, and it’s just for learning to code and practicing a bot. - - - **Are you planning to analyze Twitter data?**: No - - - **Will your app use Tweet, Retweet, Like, Follow, or Direct Message functionality?**: Yes. In the description, let Twitter know that your bot responds to users that @ mention it, and that it also periodically sends out Tweets on its own. The bot tweets and respond to Tweets with links to video game soundtracks on YouTube. - - - **Do you plan to display Tweets or aggregate data about Twitter content outside Twitter?**: No - - - **Will your product, service, or analysis make Twitter content or derived information available to a government entity?**: No - -1. Because you're not giving information to the government or consuming streaming data for analytical purposes, the application should be more straightforward. Follow the instructions on screen until you have an authenticated developer account. - -A later section in this guide shows how to create an app within the Twitter developer portal and fill out the descriptions for it. - -## Create and Clone a GitHub Repository - -As mentioned in the Before You Begin section, this tutorial records the code for the Twitter bot in a Git repository. Specifically, we first create a new repository on GitHub.com and then pull it down to your workstation. By doing this, we can populate the repository with some helpful files, like a .gitignore and a README: - -1. Visit GitHub.com, then follow the [Create a repository](https://docs.github.com/en/get-started/quickstart/create-a-repo#create-a-repository) section in GitHub's official [Create a repo](https://docs.github.com/en/get-started/quickstart/) document. When creating the repository, make these selections: - - - You can name the repository whatever you would like, but later code examples in the tutorial assume it is named `snes-soundtracks`. - - - The repository can be public or private. If you make it private, *and if you also want to use the Git command line tools*, then you need to authenticate with GitHub inside your terminal. This authentication requires a few extra steps, and these steps are outlined in the note below. For a simpler experience, it's recommended to make the repository public. - - {{< note respectIndent=false >}} -To authenticate with Github inside the terminal, see the [Authenticating with the command line](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/about-authentication-to-github#authenticating-with-the-command-line) section of GitHub's [About authentication to GitHub](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/about-authentication-to-github) document. - -Specifically, [personal access tokens](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) are the recommended method of authenticating for this tutorial. [Working with public keys](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) is not demonstrated. -{{< /note >}} - - - Enable the README file toggle. - - - Choose a license. [choosealicense.com](https://choosealicense.com/) has helpful recommendations for which license to use. This tutorial recommends the MIT License. - - - Enable the `.gitignore` toggle and choose **Node** from the available options. - - {{< note respectIndent=false >}} -A Git repository's `.gitignore` file allows you to specify certain files or file types that should not be checked into source control. Node.js projects feature some of these files. GitHub's `.gitignore` template for Node.js sets up some appropriate default files to exclude. For more information about `.gitignore` file options, see [this document from git-scm.com](https://git-scm.com/docs/gitignore). -{{< /note >}} - -1. After you finish the create repository form, the new repository appears in your browser. From this page, download a copy of the repository to your workstation: - - - **To use GitHub Desktop**, follow GitHub's [Cloning a repository from GitHub to GitHub Desktop](https://docs.github.com/en/desktop/contributing-and-collaborating-using-github-desktop/adding-and-cloning-repositories/cloning-a-repository-from-github-to-github-desktop) document. Here's an example screenshot of where the **Open with GitHub Desktop** option is in your repository: - - ![Github.com repository showing the green Code button clicked with Open in Github Desktop option highlighted](github-repo-open-in-github-desktop-highlighted.png "Github.com repository showing the green Code button clicked with Open in Github Desktop option highlighted") - - - **To use the command line**, run the `git clone` command in your terminal as follows: - - git clone https://github.com/your-github-username/snes-soundtracks.git - - Before running the command, be sure to substitute your own GitHub username in for `your-github-username`. If you chose a different name for your repository, substitute that in for `snes-soundtracks`. - - {{< note respectIndent=false >}} -If you created a private repository, remember to set up a [personal access token](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) for GitHub. When referencing a private repository, the above `git clone` command prompts for a GitHub username and password. You should enter the *personal access token* (**not** your normal GitHub password) when prompted for the password. -{{< /note >}} - -At this point, your source control is present on both your machine and on GitHub. - -## Create an App in the Twitter Developer Portal - -After your developer account has been registered, you can create an app within the developer portal. When you create the app, Twitter creates a set of keys that your Node.js project uses to authenticate with the service. This section shows how to create your Twitter app and where to store the app's keys. - -1. Create a file named `.env` inside your code repository on your workstation. Paste the following snippet into the file. - - {{< file ".env" >}} -consumer_key= -consumer_secret= -access_token= -access_token_secret= -{{< /file >}} - - Later in this section, the keys that are provided by the Twitter developer portal are recorded in this file. Your `.env` file serves as your one source of truth for these keys. The `.gitignore` for your project excludes the `.env` file from source control, so you won’t accidentally upload your keys to GitHub. - - {{< note respectIndent=false >}} -In a later section, a Node.js module named `dotenv` is used by your application code to read the secret keys in the `.env` file. -{{< /note >}} - -1. Return to the [Twitter developer portal dashboard](https://developer.twitter.com/en/portal/dashboard) in your browser. - -1. Click the **Create Project** button. - - ![Twitter developer portal with Create Project button highlighted](twitter-developer-portal-create-project-button.png "Twitter developer portal with Create Project button highlighted") - - Twitter's developer portal has two concepts that are used to manage your access to the Twitter API: *projects* and *apps*. A project contains an app. For earlier versions of the Twitter API, standalone apps were created, and projects were not required. A project and corresponding app are both required to access the v2 Twitter API, which this tutorial uses. - - {{< note respectIndent=false >}} -Review Twitter's [Developer Apps](https://developer.twitter.com/en/docs/apps/overview) documentation for more information about projects and apps. -{{< /note >}} - -1. Complete the forms that appear: - - - **Name your Project**: The name of your project needs to be unique. - - - **Which bests describes you?**: Choose **Making a bot**. - - - **Describe your new Project**: Enter a brief description of the bot's abilities: periodically tweeting links to video game soundtracks on YouTube, and responding to users that @ mention the bot. - - - **Add your App**: You may see a screen that looks like the following, which asks you to pick an app to add to the new project: - - ![Twitter developer portal: create new project form with choose app dropdown menu highlighted](twitter-create-project-form-create-new-app-option-highlighted.png "Twitter developer portal: create new project form with choose app dropdown menu highlighted") - - Because you don't have any apps yet, the dropdown menu on this page is empty. Click the **Create new App instead** button above the dropdown menu. - A text field appears that allows you to enter an app name. Enter a unique name in this field. - - {{< note respectIndent=false >}} -You may not see the dropdown menu, and the text field might appear right away instead. -{{< /note >}} - - ![Twitter developer portal: create new project form with new app name text field highlighted](twitter-create-project-form-app-name-field.png "Twitter developer portal: create new project form with new app name text field highlighted") - -1. After completing the forms, a page with secret keys appears: - - ![Twitter developer portal: API Key, API Secret Key, and Bearer Token dialog for new project and app](twitter-create-project-api-key-api-secret-key-bearer-token.png "Twitter developer portal: API Key, API Secret Key, and Bearer Token dialog for new project and app") - -1. Copy and paste the keys into your `.env` file as follows: - - - Copy the **API Key** and paste it at the end of the `consumer_key=` line. No extra spaces or quotes should be added. For example, if your API Key was `dvF6z55HXwpSE3qBVFNONV2DN`, then the line would be written like `consumer_key=dvF6z55HXwpSE3qBVFNONV2DN`. - - - Copy the **API Secret Key** and paste it at the end of the `consumer_secret=` line. As with the previous line, do not add extra spaces or quotes. - - {{< note respectIndent=false >}} -This tutorial does not need the Bearer Token, but you can record that somewhere outside of your project if you'd like to. It can also be regenerated from inside the developer portal later if you don't record it. -{{< /note >}} - -1. Close the dialog that displays the secret keys. The developer portal should now look like this: - - ![The Twitter developer portal after project and app have been created](twitter-developer-portal-after-app-created.png "The Twitter developer portal after project and app have been created") - -### Give your Twitter App Read and Write Permissions - -By default, the app that you created has Read Only permissions. Apps that are set to Read Only are not able to create Tweets, which is the primary function of the bot in this tutorial. With Read and Write permissions, your bot is able to create Tweets. Follow these instructions to give your bot Read and Write permissions: - -1. Navigate to the app (not the project) in the Twitter developer portal. The app is listed under the project in the sidebar navigation of the developer portal. - -1. Scroll down to the **App permissions** heading. This section should display your app as **Read Only**. Click on the **Edit** button in this section. - - ![Twitter developer portal: app permissions pane with edit button highlighted](twitter-developer-portal-app-permissions-edit-button-highlighted.png "Twitter developer portal: app permissions pane with edit button highlighted") - -1. In the form that appears, choose the **Read and Write** option and then click **Save**. - -1. Navigate to the **Keys and Tokens** tab at the top of the page. - -1. Scroll down to the **Access Token and Secret** pane. Click the **Generate keys** button for the access token and secret. - -1. A dialog appears that displays secret keys. Copy and paste the keys into your `.env` file as follows: - - - Copy the **Access Token** and paste it at the end of the `access_token=` line in your `.env` file, with no extra spaces or quotes. - - - Copy the **Access Token Secret** and paste it at the end of the `access_token_secret=` line in your `.env` file, with no extra spaces or quotes. - -1. Dismiss the dialog. - -## Initialize the Node Project and Install npm Dependencies - -Right now, your local repository contains four files: your `.env` secrets, a `.gitignore`, a `README.md`, and a `LICENSE` file. It does not have any Node.js project files yet. To initialize your Node.js project: - -1. Navigate to your local code repository in your terminal: - - cd snes-soundtracks - - {{< note respectIndent=false >}} -If you are using VS Code's built-in terminal and have your code repository folder open, then you do not need to run this `cd` command. This is because VS Code's terminal automatically starts inside the folder for your editor's workspace. -{{< /note >}} - -1. Run npm's `init` command: - - npm init -y - - The following output appears: - - {{< output >}} -npm init -y -Wrote to /home/username/snes-soundtracks/package.json: - -{ - "name": "snes-soundtracks", - "version": "1.0.0", - "description": "", - "main": "index.js", - "scripts": { - "test": "echo \"Error: no test specified\" && exit 1" - }, - "repository": { - "type": "git", - "url": "git+https://github.com/username/snes-soundtracks.git" - }, - "keywords": [], - "author": "", - "license": "ISC", - "bugs": { - "url": "https://github.com/username/snes-soundtracks/issues" - }, - "homepage": "https://github.com/username/snes-soundtracks#readme" -} -{{< /output >}} - - This output shows the contents of a new `package.json` file that was created by the `npm init` command. The `package.json` file contains metadata about your Node.js project. For more information about this file, see the [What is the file `package.json`](https://nodejs.org/en/knowledge/getting-started/npm/what-is-the-file-package-json/) article in the official Node.js documentation, and the [`package.json` reference](https://docs.npmjs.com/cli/v7/configuring-npm/package-json) in the npm documentation. - -1. The application code for your bot relies on a few different dependencies from npm. To install these dependencies, enter these commands in your terminal while still inside the code repository: - - npm install node - npm install node-schedule - npm install twit - npm install dotenv - - These dependencies provide everything needed to have a functional Twitter bot: - - - [node](https://www.npmjs.com/package/node): Installing the `node` package provides a local Node.js binary inside your project folder. This is installed locally so that the project doesn't depend on your system-wide version of Node.js. This makes your project more portable, because other systems that you install the project on may run a different version of Node.js. - - - [node-schedule](https://www.npmjs.com/package/node-schedule): This package is used by the tutorial to schedule new Tweets periodically. - - - [dotenv](https://www.npmjs.com/package/dotenv): This package is used to read the secrets that you stored in your `.env` file. - - - [twit](https://www.npmjs.com/package/twit): This is a Twitter API client for Node.js. - - At this point, the contents of your directory should now look like the following: - - ls -a - - {{< output >}} -. .env .gitignore README.md package-lock.json -.. .git LICENSE node_modules package.json -{{< /output >}} - -### Update Scripts in package.json - -Before we can start adding our code, we should adjust a few parts of the `package.json` file in the new Node.js project. - -1. Currently, the `scripts` section of your `package.json` should look like: - - {{< file "package.json" javascript >}} - "scripts": { - "test": "echo \"Error: no test specified\" && exit 1" - }, -{{< /file >}} - - The `scripts` section contains commands that can be used to invoke your project. For more information about this part of `package.json`, see the [`scripts` reference](https://docs.npmjs.com/cli/v7/using-npm/scripts) in the npm documentation. - - Open this file and modify the `scripts` section to look like the following snippet: - - {{< file "package.json" javascript >}} - "scripts": { - "test": "snes.js", - "start": "node ./snes.js", - "develop": "NODE_ENV=develop node snes.js" - }, -{{< /file >}} - -1. Scroll to the `main` section of the same file. This should currently look like: - - {{< file "package.json" javascript >}} - "main": "index.js", -{{< /file >}} - - Update this line so that it instead says: - - {{< file "package.json" javascript >}} - "main": "snes.js", -{{< /file >}} - -1. These changes reference an `snes.js` file that doesn’t exist yet. Create an empty placeholder file with this name in your code repository. The [`touch` command](https://en.wikipedia.org/wiki/Touch_(command)) can do this from your terminal (or you can just create the empty file with your text editor): - - touch snes.js - - {{< note respectIndent=false >}} -The `snes.js` file can be named whatever you want it to be. If you want to use a different name, just make sure that your `package.json` is updated to reference that new name. -{{< /note >}} - -## Add the Twitter Bot Application Code - -Your Node.js project is now prepared for the Twitter bot application code. Open `snes.js` in your text editor, then proceed through the next sections: - -### Reading .env Secrets, Setting Up node-schedule, and Specifying YouTube Videos - -Copy and paste this snippet into `snes.js`. After copying and pasting, **make sure to replace the `@SnesSoundtracks` string on line 3 with your own Twitter username**: - -{{< file "snes.js" javascript >}} -console.log("SNES Soundtracks booting up"); - -const twitterUsername = '@SnesSoundtracks'; - -//making sure npm run develop works -if (process.env.NODE_ENV === "develop") { - require("dotenv").config(); -}; - -//rules for node-schedule -var schedule = require("node-schedule"); -var rule = new schedule.RecurrenceRule(); -rule.dayOfWeek = 1, -rule.hour = 10; -rule.minute = 0; -rule.tz = "Etc/GMT+4"; - -//array to pull soundtracks from -var soundtrackArray = [ - "https://www.youtube.com/watch?v=85u34SUh05Y", // Legend of Zelda - "https://www.youtube.com/watch?v=85u34SUh05Y", // Chrono Trigger - "https://www.youtube.com/watch?v=UyNufyV3VCo", // Super Metroid - "https://www.youtube.com/watch?v=Y2VJeZDejtc", // Final Fantasy VI - "https://www.youtube.com/watch?v=wgUmFPnkoHU", // Super Mario World - "https://www.youtube.com/watch?v=-QsysJwzod4", // Super Street Fighter II - "https://www.youtube.com/watch?v=oRxgYC5zrV4", // Super Mario World 2: Yoshi's Island - "https://www.youtube.com/watch?v=rJJk9Zk2h_U", // Super Mario Kart - "https://www.youtube.com/watch?v=byIjMomjWkA", // Star Fox - "https://www.youtube.com/watch?v=wpchBo75N68", // Super Mario RPG: Legend of the Seven Stars - ]; -var soundtrackArrayLength = soundtrackArray.length; -{{< /file >}} - -Here's what the code does: - -- On line 1, the code logs a start up message to the console just so we know it’s running. - -- On line 7, the dotenv module is included using [the built-in Node.js `require()` function](https://nodejs.org/en/knowledge/getting-started/what-is-require/). The dotenv module's [`config()` function](https://github.com/motdotla/dotenv#config) is used on the same line to read the values from your `.env` file. The values are placed in the [`process.env` Node.js property](https://nodejs.org/dist/latest-v8.x/docs/api/process.html#process_process_env). For example, the `consumer_key` secret can be accessed inside `process.env.consumer_key` within the code. - - The `if` statement on lines 6-8 ensures this only happens when the Node.js environment is set to `develop`. This is handled by the `"develop": "NODE_ENV=develop node snes.js"` line in the `scripts` section of your package.json file. - -- On lines 11-15, we set up a few variables for the node-schedule package. These variables declare the day and time when the bot should periodically tweet. Specifically, the scheduled time is every Monday at 10 am Eastern Daylight Savings Time. A `RecurrenceRule` is created to facilitate this. - - {{< note respectIndent=false >}} -Documentation for this method of scheduling is found in the [Recurrence Rule Scheduling](https://github.com/node-schedule/node-schedule#recurrence-rule-scheduling) section of the node-schedule package's README. Creating the `RecurrenceRule` does not actually schedule the bot's tweeting function. This function is scheduled at the end of the bot's application code, in the [Scheduling Tweets](#scheduling-tweets) section. -{{< /note >}} - -- Lines 18-30 set up an array of soundtrack links for the bot to pull from at random. - -### Authenticating with Twitter and Listening for Tweets - -1. Append this snippet to the bottom of `snes.js`: - - {{< file "snes.js" javascript >}} -// ... append to bottom of file: - -// Create a Twitter client object to connect to the Twitter API -var Twit = require('twit'); - -// Pulling keys from another file -var config = require('./config.js'); -// Making a Twit object for connection to the API -var T = new Twit(config); - -// Setting up a user stream -var stream = T.stream('statuses/filter', { track: twitterUsername }); - -// Now looking for Tweet events -// See: https://dev.Twitter.com/streaming/userstreams -stream.on('tweet', pressStart); -{{< /file >}} - - We start using Twit in this section of code: - - - Line 4 includes the `twit` Twitter API client module. - - - Line 7 uses another `config.js` file to create a new configuration object. The `config.js` file is created in the next step of this section. - - - Line 9 uses [JavaScript's `new` operator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new) to create an instance of the Twitter API client. The configuration object from line 7 is passed to the *constructor function* for the new instance. - - This instance is stored in the variable `T`. From now on, when we want to use something from the `twit` module, we simply reference the `T.propertyName` syntax. This syntax allows us to access any property, field, or method we need from the API client's library. - - - Line 12 sets up a stream that monitors for Tweet mentions of your Twitter username from other Twitter users while the code is running. - - - Line 16 creates an event listener with the `stream.on` method. The `stream.on` method accepts two arguments: - - - A string parameter that represents the type of [event](https://github.com/ttezel/twit#event-tweet) that should be listened for. - - - Another function that handles the events from the stream. The function that handles the events is called `pressStart`. The `pressStart` function is defined in the next section of this tutorial. - -1. In your code repository, make a file named `config.js` and paste this snippet into it: - - {{< file "config.js" javascript >}} -module.exports = { - consumer_key: process.env.consumer_key, - consumer_secret: process.env.consumer_secret, - access_token: process.env.access_token, - access_token_secret: process.env.access_token_secret, -}; -{{< /file >}} - - This file exports an object that matches the configuration object accepted by the constructor function in the `twit` module. Specifically, `twit` expects a configuration object with the following form, as described in [the `twit` README on GitHub](https://github.com/ttezel/twit#var-t--new-twitconfig): - - ``` - { - consumer_key: '...', - consumer_secret: '...', - access_token: '...', - access_token_secret: '...' - } - ``` - - In order to generate the configuration object, this file reads the values encoded in the `process.env` property by the dotenv module. - -### Replying to Tweets - -Append this snippet to the bottom of `snes.js`: - -{{< file "snes.js" javascript >}} -// ... append to bottom of file: - -function pressStart(tweet) { - - var id = tweet.id_str; - var text = tweet.text; - var name = tweet.user.screen_name; - - let regex = /(please)/gi; - - - let playerOne = text.match(regex) || []; - let playerTwo = playerOne.length > 0; - - //this helps with errors, so you can see if the regex matched and if playerTwo is true or false - console.log(playerOne); - console.log(playerTwo); - - - // checks text of tweet for mention of SNESSoundtracks - if (text.includes(twitterUsername) && playerTwo === true) { - - // Start a reply back to the sender - var soundtrackArrayElement = Math.floor(Math.random() * soundtrackArrayLength); - var replyText = ("@" + name + " Here's your soundtrack: " + soundtrackArray[soundtrackArrayElement]); - - // Post that tweet - T.post('statuses/update', { status: replyText, in_reply_to_status_id: id }, gameOver); - - } else { - console.log("uh-uh-uh, they didn't say the magic word."); - }; - - function gameOver(err, reply) { - if (err) { - console.log(err.message); - console.log("Game Over"); - } else { - console.log('Tweeted: ' + reply.text); - } - }; -} -{{< /file >}} - -This section of code defines a `pressStart` function that's called when another Twitter user mentions the bot. It contains a few local variables, a bit of logic, and another callback function that must be included in the `T.post` method: - -- On line 3, the `pressStart` function is defined. It takes a `tweet` object as a parameter. This represents a Tweet that another Twitter user has mentioned the bot in. The `tweet` object has tons of data attached to it, and the Twit client helps us parse through this data. - -- Lines 5-7 parse three variables from this object: the id of the Tweet, the text of the Tweet, and the username of the person who wrote the Tweet. We need those three variables in order to respond as a comment to the original Tweet. - -- Lines 9-13 define and use a [*regular expression*](https://en.wikipedia.org/wiki/Regular_expression) (abbreviated *regex*) to analyze the text of the Tweet object. This regex looks for a keyword that you want to be present in Tweet mentions. If it matches the text of the Tweet, then the bot is activated and creates a reply Tweet. The tutorial code checks for the word `please`, but you can choose another word if you prefer. As long as other Twitter users are polite and include this word, they’ll get a random soundtrack in response. - - - The `gi` characters at the end of the `/(please)/gi` regex definition represent two different options that are used when executing the expression: - - - By default, a regex only looks for the first match that it finds in the text string that it examines. The `g` option tells the regex to return all matches throughout the string. For example, if a Tweet includes the word `please` three times, then the regex in this tutorial returns all three matches. - - - The `i` option ignores capitalization. - - - The `playerOne` variable is set to an array of the regex's matches. If there are no matches, then it is set to an empty array. - - - The `playerTwo` variable is a boolean that verifies whether the array `playerOne` has anything in it. In other words, `playerTwo` is set to true if the regex matched with the Tweet text. - -- The `if` block on line 21-30 composes the reply Tweet: - - - The `if` statement requires that the Tweet text contains the bot’s name and the keyword from the regex (via the `playerTwo` boolean). - - - If both of these conditions are true, then we proceed with creating the reply Tweet. The variable `replyText` is a text string created with a random element of the soundtrack list array, the Twitter username of the person being replied to, and a short message. - - - On line 28, the [`T.post` method](https://github.com/ttezel/twit#tpostpath-params-callback) sends a POST request to [the `statuses/update` Twitter API endpoint](https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/api-reference/post-statuses-update). - - - An object with a `status` property and an `in_reply_to_status_id` property is passed to the `T.post` method. `status` is the actual text to be posted in the reply Tweet. In our code, it is set to the `replyText` variable. `in_reply_to_status_id` specifies a unique identifier of a Tweet from Twitter’s API. This allows Twit to identify which Tweet the bot replies to as a comment. In our code, it is set to the `id` variable from the beginning of the `pressStart` function. - - - A `gameOver` callback function is also passed to the `T.post` method, and this function is defined on lines 34-41. This function prints the reply Tweet to your console, or it prints any errors that the Twitter API may have returned from the POST request. - - {{< note respectIndent=false >}} -Where the `gameOver` function is passed to `T.post`, you could use an anonymous function instead, and it would do the same thing. The tutorial code defines it separately for better readability. -{{< /note >}} - - - The `else` block on lines 30-32 is invoked if the Tweet mention doesn't say `please`. It logs [a quote from Jurassic Park](https://youtu.be/RfiQYRn7fBg) to the console, just so we can see why the bot didn't tweet back. - -### Scheduling Tweets - -Append this snippet to the bottom of `snes.js`: - -{{< file "snes.js" javascript >}} -// ... append to bottom of file: - -function pressSelect() { - var now = new Date(); - const dateOptions = { weekday: 'long', year: 'numeric', month: 'long', day: 'numeric' }; - const timeOptions = { timeZoneName: 'short' }; - const dateString = now.toLocaleDateString("en-US", dateOptions) + " at " + now.toLocaleTimeString("en-US", timeOptions); - - var soundtrackArrayElement = Math.floor(Math.random() * soundtrackArrayLength); - var weeklyText = soundtrackArray[soundtrackArrayElement] + " Here's your soundtrack for " + dateString; - T.post('statuses/update', { status: weeklyText }, gameOver2); - - function gameOver2(err, reply) { - if (err) { - console.log(err.message); - console.log("Game Over"); - } else { - console.log('Tweeted: ' + reply.text); - } - } -} - -const job1 = schedule.scheduleJob(rule, pressSelect); -{{< /file >}} - -This section of code defines a `pressSelect` function that sends new Tweets with a link to a soundtrack. It then schedules the function to be called periodically: - -- Lines 4-10 compose a `weeklyText` string, which is similar to the `replyText` variable from the `pressStart` function. The text is slightly changed to be an original Tweet, rather than a comment on a different Tweet. It also pulls an element from the same soundtrack list array. - -- On line 11, The `T.post` method is invoked with the same `statuses/update` API endpoint. There is no `in_reply_to_status_id` property passed, because the function composes an original Tweet and not a reply. - -- The `gameOver` callback function is also present on lines 13-20, but it has been renamed to `gameOver2` to avoid name conflicts. Since `gameOver` and `gameOver2` are local variables within their respective functions, there shouldn’t be any issues. But, we've changed the name just to be safe. - -- After the `pressSelect` definition, line 23 invokes [the node-schedule module's `scheduleJob` function](https://github.com/node-schedule/node-schedule#jobs-and-scheduling) and stores the result in the `job1` variable. This is the last step needed to set up periodic scheduled Tweets. This function accepts two arguments: - - - The `rule` variable created earlier in the code, which describes when the scheduled job should be run (once a week on Monday). - - - The `pressSelect` function. - -## Commit the Code and Push to GitHub - -Now that the bot code has been added to your workstation's copy of the repository, we should make a new *commit*. A commit in Git records your file changes in the version control history. After making the new commit, you can then **push** it to your repository on GitHub. - -{{< note >}} -A commit that you make locally is not automatically synced to GitHub. The push operation demonstrated in this tutorial needs to be done manually. -{{< /note >}} - -In the next two sections, this tutorial demonstrates two methods for creating a commit and pushing it to GitHub. Use the one you prefer: [Commit and Push with GitHub Desktop](#commit-and-push-with-github-desktop), or [Commit and Push with the Git Command Line](#commit-and-push-with-github-command-line). - -### Commit and Push with GitHub Desktop - -1. Open GitHub Desktop on your workstation. The new files that you created are shown under the **Changes** tab on the left, and they are automatically selected for the new commit. - - ![GitHub Desktop - Commit to main](github-desktop-commit-to-main.png "GitHub Desktop - Commit to main") - - {{< note respectIndent=false >}} -Not all of the files that are in your Node.js project, like the `node_modules` folder, or the `.env` secrets file, are shown. This is because they are excluded by the `.gitignore` file. -{{< /note >}} - -1. Below the changed files, enter a brief summary message for the new commit. For example: `Add Twitter bot application code`. - -1. Press the **Commit to main** button below the summary and description fields. This creates the new commit on the *main* branch of your repository, which is the default branch name for repositories created on GitHub. - - {{< note respectIndent=false >}} -Branches in Git allow you to work on new features for your projects without altering the files in your other branches. -{{< /note >}} - -1. After creating the commit, the **Changes** tab is empty: - - ![GitHub Desktop - after commit with push origin option highlighted](github-desktop-push-origin.png "GitHub Desktop - after commit with push origin option highlighted") - -1. Click on the **Push origin** that appears in the top navigation. This pushes the new commit to your GitHub repository. - - {{< note respectIndent=false >}} -*Origin* is how Git is referring to the remote repository on GitHub. This is a naming convention for Git. -{{< /note >}} - -1. If you visit your repository on GitHub, the new files for your Node.js project are present. - -### Commit and Push with the Git Command Line - -1. Return to your local code repository in your terminal, then run: - - git status - - The following output appears: - - {{< output >}} -On branch main -Your branch is up to date with 'origin/main'. - -Untracked files: - (use "git add ..." to include in what will be committed) - config.js - package-lock.json - package.json - snes.js -{{< /output >}} - - Your new application files are listed under the `Untracked files` line, which means that they don't exist yet in Git's version history. - - {{< note respectIndent=false >}} -Not all of the files that are in your Node.js project, like the `node_modules` folder, or the `.env` secrets file, are shown. This is because they are excluded by the `.gitignore` file. -{{< /note >}} - -1. Before you can commit the files, you need to *stage* them. Staging tells Git that you want to include the files in the next commit that you make. To stage your files, run: - - git add . - - {{< note respectIndent=false >}} -The period at the end of this command tells git to add all the files in your current directory to the stage. -{{< /note >}} - -1. Then run the `status` command again: - - git status - - The output now appears as: - - {{< output >}} -On branch main -Your branch is up to date with 'origin/main'. - -Changes to be committed: - (use "git restore --staged ..." to unstage) - new file: config.js - new file: package-lock.json - new file: package.json - new file: snes.js -{{< /output >}} - - The output above mentions that you are on the `main` branch, which is the default branch name for repositories created on GitHub. - - {{< note respectIndent=false >}} -Branches in Git allow you to work on new features for your projects without altering the files in your other branches. -{{< /note >}} - -1. Git now knows to include the new files in your next commit. To make the commit, run: - - git commit -m "Add Twitter bot application code" - - This output appears: - - {{< output >}} -[main e5d6839] Add Twitter bot application code - 4 files changed, 599 insertions(+) - create mode 100644 config.js - create mode 100644 package-lock.json - create mode 100644 package.json - create mode 100644 snes.js - {{< /output >}} - - 1. Next, push the new commit to your repository on GitHub: - - git push origin main - - {{< note respectIndent=false >}} -*Origin* is how Git is referring to the remote repository on GitHub. This is a naming convention for Git. -{{< /note >}} - - This output appears: - - {{< output >}} -Enumerating objects: 7, done. -Counting objects: 100% (7/7), done. -Delta compression using up to 12 threads -Compressing objects: 100% (6/6), done. -Writing objects: 100% (6/6), 7.57 KiB | 1.08 MiB/s, done. -Total 6 (delta 0), reused 0 (delta 0) -To https://github.com/your-github-username/snes-soundtracks.git - 735ecd1..e5d6839 main -> main -{{< /output >}} - -1. If you visit your repository on GitHub, the new files for your Node.js project are present. - -## Run the Code - -To test your code for the first time and ensure it works, type `npm run develop` into the terminal: - - npm run develop - -If successful, the following output appears: - -{{< output >}} -> snes-soundtracks@1.0.0 develop -> NODE_ENV=develop node snes.js - -SNES Soundtracks booting up -{{< /output >}} - -If any errors appear, they are likely related to an authentication failure with the Twitter API. If this happens, then your `.env` file might have syntax errors, or your secret keys may have been incorrectly pasted into it. Review the [Troubleshooting `consumer_key` Twit Config Error](#troubleshooting-consumer_key-twit-config-error) and [Troubleshooting Twitter Error 401 Unauthorized](#troubleshooting-twitter-error-401-unauthorized) sections for solutions. - -### Test the Tweet Function - -At this point, the bot is running, and if you wait long enough, then it should send a scheduled Tweet. We can also test `pressSelect` on its own and make sure the tweeting function works, To do this, you need to modify the code slightly. - -1. Comment out the last line of the code in `snes.js` and call `pressSelect` directly: - - {{< file "snes.js" javascript >}} -// ... modify end of file as follows: - -// const job1 = schedule.scheduleJob(rule, pressSelect); - -pressSelect(); -{{< /file >}} - - By doing this, the tweeting function runs immediately, and you don’t have to wait until the next scheduled Tweet. - -1. Type `CTRL-c` to interrupt and stop the bot. Then run the code again: - - npm run develop - - If successful, output like the following output appears: - - {{< output >}} -> snes-soundtracks@1.0.0 develop -> NODE_ENV=develop node snes.js - -SNES Soundtracks booting up -Tweeted: https://t.co/Y2AOzzkUm0 Here's your soundtrack for Tuesday, June 1, 2021 at 10:59:19 AM UTC -{{< /output >}} - - {{< note respectIndent=false >}} -Note that Twitter has automatically shortened the link to the soundtrack in the Tweet. -{{< /note >}} - - If you don't see the above output, you might instead see this error: `Read-only application cannot POST.`. If so, then your app's permissions are set to Read Only in the Twitter developer portal. Revisit the earlier [Give your Twitter App Read and Write Permissions](#give-your-twitter-app-read-and-write-permissions) section and then return to this section. - -### Test Tweet Replies - -Lastly, you can test the way the bot responds to other users: - -1. Undo the code changes you made in the [Test the Tweet Function](#test-the-tweet-function) section: remove the `pressSelect();` line that was added, uncomment the last line, and save the file. - -1. Type `CTRL-c` to interrupt and stop the bot. Then run the code again: - - npm run develop - -1. Log in to another Twitter account and tweet at your bot, including the keyword (`please`). You should be able to see some action in the terminal that tells you it’s working, followed by the response on Twitter. - -## Host the Code - -You could certainly let this code run for a long while from your local machine, but it’s better if you host it somewhere else. This section shows how to create a compute instance on Linode, install the bot on it, and then run it. - -### Create a Linode - -1. Visit the [Linode Cloud Manager](https://cloud.linode.com). If you haven't created any Linodes before, the home page looks like this: - - ![Linode home screen](https://lh3.googleusercontent.com/JFNpdFMCe9A37beAwtxazN-zqcSr88Ff457bnQhbQpkQJILfqAv7g0bR_CQ6SxMu8EfKgIcaqTGuZvPTTI2hOb6dYyi3CyLMubEKOwFEZMkCaByjpk83L2o0c4W8GTwE4VPSodE-) - -1. Follow the [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guide. When creating your instance, use the following options: - - - Pick Ubuntu 20.04 as your Linux distribution. - - - Select the region closest to you. - - - Choose Nanode (the smallest option) as your Linode plan. It’s only $5 a month and has more than enough space and RAM for your Twitter bot and any other projects you’d want on it. - - - Pick a long, unique, and complex root user password. The root user for your server can take any administrative action on the system. Using a strong root password is important to prevent malicious attackers from accessing your system. - - - When it asks for a Linode label, name it after your project. - -1. After you create the Linode, it takes a few moments to boot up. Once it boots up it’s time to get inside and clean up a bit. - -### Log In and Secure the Server - -1. To log into the server, follow the [Connect to Your Linode via SSH](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#connect-to-the-instance) section of our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide. If you do not have access to an SSH client, or if SSH connections are firewalled on your local network, you can also opt to use [the Lish console](https://techdocs.akamai.com/cloud-computing/docs/access-your-system-console-using-lish) from the Cloud Manager in your web browser. To do so, follow the [Use a Web Browser](https://techdocs.akamai.com/cloud-computing/docs/access-your-system-console-using-lish#through-cloud-manager-weblish) instructions in our Lish guide. - - {{< note respectIndent=false >}} -Our [Connecting to a Remote Server Over SSH](/cloud/guides/connect-to-server-over-ssh) guide series has a few other options for SSH clients, like SSH extension for the Chrome web browser. -{{< /note >}} - - When logging in for the first time, you use the `root` username and the root password that you set when creating the Linode. However, it's important to create a non-root-user with limited permissions to run your programs from. - - {{< note type="alert" respectIndent=false >}} -If you were to run your programs as root, and if they were to be compromised by someone malicious, then the rest of your server could be compromised. If your server isn't running anything else, then it may not be important to you. However, an attacker could install malicious programs that target other people's servers, like a botnet script that sends denial-of-service attacks. -{{< /note >}} - -1. To create a non-root-user, follow the [Add a Limited User Account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account) section of our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide. This guide assumes that the name of the new user is `tutorialbotuser`, but you can name it whatever you'd like. The instructions in this section also show how to give your user `sudo` privileges, so it is still able to perform software updates and other administrative tasks. - - {{< note respectIndent=false >}} -Here's a video that also shows how to create a limited user: [Tech Republic: How to create a new user with admin privileges](https://youtu.be/fDHHKR0nVQg). -{{< /note >}} - -1. After you have created the limited user, log out of your current SSH (or Lish) session by running the `exit` command. Then, log back in as the new user. - - {{< note respectIndent=false >}} -The rest of the steps in the [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide are optional for this tutorial, but they are still recommended if you intend to leave your server running. -{{< /note >}} - -### Perform Software Updates and Prepare for the Bot - -1. Once you’re logged in with your limited user, you should run `sudo apt update` to check for updates available, and then `sudo apt dist-upgrade.` - -1. Node.js and NPM are not preinstalled on Ubuntu 20.04, so you need to install those in order to run the bot. As on your workstation, the recommended way is to use nvm: - - 1. On the Linode, run the nvm installer: - - wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash - - 1. Log out of the Linode and then log back in again. This is needed to start using the nvm command. - - 1. Install the latest Node.js and npm via nvm: - - nvm install --lts - -The server is now ready to run the bot, but you still need to upload the bot's code to the server. We present two ways to do this: - -- **Filezilla**: You can transfer your code to the Linux instance from your workstation, via the [Filezilla](/cloud/guides/filezilla) GUI desktop application. Because you're using a desktop application, this can be a little more user-friendly for beginners. Follow the [Upload the Bot Using Filezilla](#upload-the-bot-using-filezilla) section to do this. - -- **git clone**: You can directly clone your repository on GitHub to your Linode with the Git command line tool. Follow the [Upload the Bot with Git Clone](#upload-the-bot-with-git-clone) section to do this. - -### Upload the Bot Using Filezilla - -1. Install the Filezilla client by downloading the installer for your workstation from their [downloads page](https://filezilla-project.org/download.php). - -1. Here's how Filezilla looks once you open it: - - ![The opening screen of Filezilla](https://lh5.googleusercontent.com/Wl8STNGN-_2A-4jBHrTZS0evnCwnt_Ts_vuSqFl9NTfzhG2pGHpZb9rvjpcL80QXLZ1NeKz5xzE2XzepPWU6LXZ2-VVnbeYpJ7GvXyP6xfoSgiT3EROD2Wogm3NEcWfheHnYDAop) - -1. Inside Filezilla, you can see input fields for **Host**, **Username**, **Password**, and **Port**. Set values for these fields as follows: - - | Field | Value | - |-------|-------| - | Host | The IP Address for your Linode. Use the [Find your Linode's IP Address](https://techdocs.akamai.com/cloud-computing/docs/managing-ip-addresses-on-a-compute-instance) guide to locate your IP address in the Cloud Manager. | - | Username | This should be the name for the limited non-root-user you created. | - | Password | The password for the limited non-root-user you created. | - | Port | This should be set to 22. | - -1. Upon connecting, you’ll get a message saying the key is unknown. This is normal, but you should double-check that the IP address is correct. If it is, check the box to remember your Linode in Filezilla. - -1. The left half of the Filezilla window shows your local files, and the right half of the window shows your Linode's files. On the left side, go to the directory where you’re keeping the files for your Twitter bot. You’ll need the `.env` file, `config.js` file, `package.json` file, `package-lock.json` file, `node_modules` folder, and `snes.js` application code file for your bot. The files we're moving over are highlighted in the following picture. - - ![An image of the file directory containing the files for a twitter bot. .env, config.js, snes.js, and package.json are highlighted](https://lh4.googleusercontent.com/LiCT5nHxonGW71MJ2tbk_FScrHkkhIPuyBDLq74jxtfeW5YE9yGk-dBlAvlSvQaeBC5NPpiWv-VyV4YCFbrQEZHRcHatviZSuplcSOwfHj84yRMNmZXwc96OlCXK_B9MSqtFkgp7) - -1. By default, the right side of the FileZilla window shows the files in the home directory of your Linode's user. This is also where we want to install the bot. Once you highlight the files you want to transfer on the left side, click and drag them to the right. - -After finishing the file transfer, proceed to the [run the bot](#run-the-bot-on-your-linode) section. - -### Upload the Bot with Git Clone - -1. Git is preinstalled on Ubuntu 20.04, but you still need to configure it like you did on your workstation. While inside your SSH or Lish connection to your Linode, follow the [Configure Git](/cloud/guides/how-to-configure-git#configure-git) section of our [Getting Started with Git](/cloud/guides/how-to-configure-git) guide. When doing this, you don't need to set the `core.editor` option, but it is important to set your username and email for Git. - -1. To clone your GitHub repository to your Linode, run the `git clone` command from your SSH or Lish connection. Be sure to substitute your own GitHub username in for `your-github-username` and the name you chose for the repository for `snes-soundtracks` before running the command: - - git clone https://github.com/your-github-username/snes-soundtracks.git - - {{< note respectIndent=false >}} -If you previously created a private GitHub repository, then the above command prompts for your password. You should enter the *personal access token* (and not your normal GitHub password) that you created when setting up [command line authentication for GitHub](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/about-authentication-to-github#authenticating-with-the-command-line) earlier. -{{< /note >}} - -1. Cloning the repository does not install the NPM dependencies for your project. This is because they were contained in the `node_modules` folder of your project, which was excluded from version control in your `.gitignore` file. The required dependencies are still outlined in your `package.json`, which we can use to reinstall them on your Linode. Navigate to the new cloned repository and then install them: - - cd snes-soundtracks - npm install - - The `npm install` command looks at your package.json and installs every dependency that it finds into the `node_modules` folder. - -1. The `.env` file was also excluded from version control, so it needs to be copied to your Linode separately. On your workstation, open your local `.env` and copy its contents to your clipboard. Then inside the code repository on the Linode, create a new `.env` file. You can use [the `nano` command line text editor](/cloud/guides/use-nano-to-edit-files-in-linux) to do this: - - nano .env - - The `nano` editor's interface looks like this when first opened: - - ![nano text editor interface](nano-empty-env-file.png "nano text editor interface") - -1. Paste the contents of your clipboard into the editor. - -1. Type `CTRL-o` to save the file, and hit `Enter` when the editor prompts you to confirm the `.env` filename. - -1. Type `CTRL-x` to exit the editor. - -1. Remove file system read and write [file permissions](/cloud/guides/modify-file-permissions-with-chmod) for groups and other users on the Linode for the `.env` file: - - chmod go-rw .env - - Doing this ensures that your secret keys can't be read by other users on the system. - -After cloning the repository and installing the dependencies, proceed to the [run the bot](#run-the-bot-on-your-linode) section. - -### Run the Bot on your Linode - -Once you’ve transferred the files, you can use your SSH connection or Lish console to run the bot in the same way as on your workstation: - - npm run develop - -However, the bot does not stay running if you close your SSH connection (for example, if you put your workstation to sleep), or if you need to perform some other action in the Lish console. - -To keep your bot running uninterrupted, you can start a Screen session. [Screen](/cloud/guides/using-gnu-screen-to-manage-persistent-terminal-sessions) is a useful tool for creating terminal sessions that stay alive and that can be accessed from any SSH or Lish connection. It's also preinstalled on Ubuntu 20.04, so you can start using it immediately: - -1. If you ran `npm run develop` on your Linode, type `CTRL-c` to stop the bot. - -1. From your Linode, run: - - screen - -1. This creates a new Screen session. A welcome message for the new session is presented. Close this message by hitting `Enter` on your keyboard. - -1. You are now in your Screen session. Run the bot again: - - npm run develop - -1. Then, *detach* from the Screen session by typing `CTRL-a`, followed by the `d` key. This brings you back to your previous terminal history. After detaching, a message like this appears: - - {{< output >}} -[detached from 38905.pts-0.localhost] -{{< /output >}} - -1. Your bot continues to run, and it runs even if you close your SSH or Lish connection. You can later return to your bot from the same SSH or Lish connection, or from a new connection. This command *reattaches* to your screen session: - - screen -r - -1. If you want to halt your bot, reattach to the screen session, then type `CTRL-c`. - -## Troubleshooting - -### Troubleshooting `consumer_key` Twit Config Error - -When starting the bot, you may see an error like this: - -{{< output >}} -> snes-soundtracks@1.0.0 develop -> NODE_ENV=develop node snes.js - -SNES Soundtracks booting up -/home/tutorialbotuser/snes-soundtracks/node_modules/twit/lib/twitter.js:511 - throw new Error(err_msg) - ^ - -Error: Twit config must include `consumer_key` when using user auth. - at /home/tutorialbotuser/snes-soundtracks/node_modules/twit/lib/twitter.js:511:13 - at Array.forEach () - at Twitter._validateConfigOrThrow (/home/tutorialbotuser/snes-soundtracks/node_modules/twit/lib/twitter.js:508:17) - at new Twitter (/home/tutorialbotuser/snes-soundtracks/node_modules/twit/lib/twitter.js:58:8) - at Object. (/home/tutorialbotuser/snes-soundtracks/snes.js:26:9) - at Module._compile (node:internal/modules/cjs/loader:1109:14) - at Object.Module._extensions..js (node:internal/modules/cjs/loader:1138:10) - at Module.load (node:internal/modules/cjs/loader:989:32) - at Function.Module._load (node:internal/modules/cjs/loader:829:14) - at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:76:12) - at node:internal/main/run_main_module:17:47 -{{< /output >}} - -This Twit error about your `consumer_key` means there may be a syntax error in your `.env` file. In your `.env`, ensure there are no spaces between the variable, equals sign, and key itself in your .env file. As an example, here's what your `.env` might look like when correctly formatted (your keys are different): - -{{< file ".env" >}} -consumer_key=rA8yWhr4ZnuWoKGDmmdbhF4Su -consumer_secret=BVMRjkQRTzlkBRC8qVZQKkWsEIEzPE9XAV1egY2D86yrQqadXH -access_token=1399573085406306304-PkDdzGOQu6ikxbwbhpmsLoP4W87jaE -access_token_secret=gf6ohLUT06RqcvfB4H6qQRZstcc6UgkouszIiLxHLGtGj -{{< /file >}} - -If the error persists after correcting any syntax issues, you may have copied your keys incorrectly. Review the [Troubleshooting Twitter Error 401 Unauthorized](#troubleshooting-twitter-error-401-unauthorized) section for further guidance. - -### Troubleshooting Twitter Error 401 Unauthorized - -When starting the bot, you may see an error like this: - -{{< output >}} -> snes-soundtracks@1.0.0 develop -> NODE_ENV=develop node snes.js - -SNES Soundtracks booting up -node:events:342 - throw er; // Unhandled 'error' event - ^ - -Error: Bad Twitter streaming request: 401 - at Object.exports.makeTwitError (/home/tutorialbotuser/snes-soundtracks/node_modules/twit/lib/helpers.js:74:13) - at Request. (/home/tutorialbotuser/snes-soundtracks/node_modules/twit/lib/streaming-api-connection.js:96:29) - at Request.emit (node:events:377:35) - at Gunzip. (/home/tutorialbotuser/snes-soundtracks/node_modules/request/request.js:1076:12) - at Object.onceWrapper (node:events:471:28) - at Gunzip.emit (node:events:365:28) - at endReadableNT (node:internal/streams/readable:1312:12) - at processTicksAndRejections (node:internal/process/task_queues:83:21) -Emitted 'error' event on StreamingAPIConnection instance at: - at Request. (/home/tutorialbotuser/snes-soundtracks/node_modules/twit/lib/streaming-api-connection.js:99:14) - at Request.emit (node:events:377:35) - [... lines matching original stack trace ...] - at processTicksAndRejections (node:internal/process/task_queues:83:21) { - code: null, - allErrors: [], - twitterReply: '\\n\\n\\nError 401 Unauthorized\n' + - '\n' + - '\n' + - '

HTTP ERROR: 401

\n' + - "

Problem accessing '/1.1/statuses/filter.json'. Reason:\n" + - '

    Unauthorized
\n' + - '\n' + - '\n', - statusCode: 401 -} -{{< /output >}} - -This error indicates that incorrect keys were copied into your `.env` file. You may have also associated the `.env` key names with the wrong credential from the Twitter developer portal. Review the table below for the correct associations between the `.env` file and the Twitter developer portal credentials. - -| .env Key Name | Twitter Developer Portal Credential Name | -|---------------|------------------------------------------| -| `consumer_key` | API Key | -| `consumer_secret` | API Secret Key | -| `access_token` | Access Token | -| `access_token_secret` | Access Token Secret | - -If you're not sure whether your keys are correct, you can regenerate them inside the Twitter developer portal. See the [Regenerate API Keys and Tokens](https://developer.twitter.com/en/docs/authentication/guides/authentication-best-practices) section of Twitter's documentation for instructions. - -### Troubleshooting Twitter Read Only Application Cannot POST Error - -When your bot attempts to tweet, you may see an error like this in its logs: - -{{< output >}} -Read-only application cannot POST. -{{< /output >}} - -This error indicates that your app's permissions are set to Read Only in the Twitter developer portal. Follow the instructions in the [Give your Twitter App Read and Write Permissions](#give-your-twitter-app-read-and-write-permissions). This section shows how to switch to Read and Write permissions and how to regenerate your Access Token and Access Token Secret. \ No newline at end of file diff --git a/docs/guides/development/nodejs/use-nightmarejs-to-automate-headless-browsing/index.md b/docs/guides/development/nodejs/use-nightmarejs-to-automate-headless-browsing/index.md deleted file mode 100644 index d629b29db01..00000000000 --- a/docs/guides/development/nodejs/use-nightmarejs-to-automate-headless-browsing/index.md +++ /dev/null @@ -1,158 +0,0 @@ ---- -slug: use-nightmarejs-to-automate-headless-browsing -title: 'Use Nightmare.js to Automate Headless Browsing' -description: 'Automate browsing tasks with Nightmare.js, a high-level browser automation library.' -og_description: 'Nightmare.js is an automated, headless browsing tool that can be configured to self-navigate websites, automate data scraping, and quicken QA.' -authors: ["Nashruddin Amin"] -contributors: ["Nashruddin Amin"] -published: 2017-10-09 -keywords: ["nightmare.js", " node.js", " headless browser", " automation"] -tags: ["automation", "ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Nightmare.js Homepage](http://www.nightmarejs.org/)' - - '[Nightmare.js Github Repository](https://github.com/segmentio/nightmare)' -audiences: ["intermediate"] -concentrations: ["Scripting, Automation, and Build Tools"] -languages: ["javascript"] -aliases: [] ---- - -![Use Nightmare.js to Automate Headless Browsing](nightmarejs-automate-headless-browsing-title.jpg "Use Nightmare.js to Automate Headless Browsing") - -**Nightmare.js** is a high-level browser automation library, designed to automate browsing tasks for sites that don't have APIs. The library itself is a wrapper around [Electron](https://electron.atom.io/), which Nightmare.js uses as a browser to interact with web sites. This guide helps you install Nightmare.js on Ubuntu 16.04 and run automation scripts without the need for a graphical user interface. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's hostname and timezone. - -2. This guide will use `sudo` wherever possible. Complete the sections of our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) to create a standard user account, harden SSH access and remove unnecessary network services. - -3. Update your system: - - sudo apt-get update && sudo apt-get upgrade -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. - -{{< /note >}} - -## Install Node.js - -The Ubuntu 16.04 repository is slower to release recent versions of Node.js. Install the most recent available version through the NodeSource PPA (formerly Chris Lea's Launchpad PPA). - -1. Install the NodeSource PPA: - - curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - - - {{< note respectIndent=false >}} -This command fetches the latest version of Node.js 6. To install a [specific version](https://nodejs.org/en/download/releases/), replace the `6.x` in this example. - -{{< /note >}} - -2. Install Node.js and NPM with the following command: - - sudo apt-get install -y nodejs - -3. Confirm that Node.js is successfully installed: - - node --version - -4. Check that the NPM command-line tool is successfully installed as well: - - npm --version - -## Install Nightmare.js - -To avoid installing the Node packages for the system globally, install Nightmare.js in a specific directory. This examples creates a `automation` directory within the current user's home directory as the base the project. - -1. Create and switch to the `automation` directory: - - mkdir ~/automation && cd ~/automation - -2. Initialize an NPM project. NPM prompts you to provide a name, repository, and other details for the project. Accept the default values or assign whatever names your want. To accept the defaults automatically, add the `-f` force flag to this example: - - npm init - -3. Install Nightmare.js: - - npm install --save nightmare - -## Create and Run the Automation Script - -Nightmare.js is an NPM module, so it can be imported from within a Node.js script. Use these examples to write a simple script that will search Linode's documentation for guides about Ubuntu. - -1. Nightmare.js uses the Electron browser and requires an X server. Install `xvfb` and its dependencies so that you can run graphical applications without display hardware: - - sudo apt-get install -y xvfb x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-apps clang libdbus-1-dev libgtk2.0-dev libnotify-dev libgnome-keyring-dev libgconf2-dev libasound2-dev libcap-dev libcups2-dev libxtst-dev libxss1 libnss3-dev gcc-multilib g++-multilib - -2. Create `linode.js` inside the automation directory and add the following: - - {{< file "~/automation/linode.js" javascript >}} -const Nightmare = require('nightmare'); -const nightmare = Nightmare({show: true}); - - -nightmare - .goto('https://www.linode.com/docs') - .insert('.ais-SearchBox-input', 'ubuntu') - .click('.ais-SearchBox-submit') - .wait('.ais-Hits-list') - .evaluate(function() { - let searchResults = []; - - const results = document.querySelectorAll('a.c-search__result__link'); - results.forEach(function(result) { - let row = { - 'title':result.innerText, - 'url':result.href - } - searchResults.push(row); - }); - return searchResults; - }) - .end() - .then(function(result) { - result.forEach(function(r) { - console.log('Title: ' + r.title); - console.log('URL: ' + r.url); - }) - }) - .catch(function(e) { - console.log(e); - }); - -{{< /file >}} - - -3. Run the script: - - xvfb-run node linode.js - - The script visits the [Linode docs](/cloud) page, enters 'Ubuntu' into the input box, and clicks the submit button. It then waits for the results to load and prints the url and title each entry on the first page of results. - - The output will resemble the following: - - Title: How to Install a LAMP Stack on Ubuntu 16.04 - URL: https://www.linode.com/docs/web-servers/lamp/install-lamp-stack-on-ubuntu-16-04 - Title: Install and Configure MySQL Workbench on Ubuntu 16.04 - URL: https://www.linode.com/docs/databases/mysql/install-and-configure-mysql-workbench-on-ubuntu - Title: Install MongoDB on Ubuntu 16.04 (Xenial) - URL: https://www.linode.com/docs/databases/mongodb/install-mongodb-on-ubuntu-16-04 - ... - -## Add a Cron Job to Run the Automation Script - -This example automates the script to run once every hour. It changes to the `~/automation/` directory, runs the scraping script, and saves the output to a file with a unique filename that includes the date and time it ran. - -For more information about using Cron, see our [Schedule Tasks with Cron](/cloud/guides/schedule-tasks-with-cron) guide. - -1. Open the crontab file: - - crontab -e - -2. Add the following line to the end of the file: - - {{< file "crontab" cron >}} -0 * * * * cd ~/automation && xvfb-run node linode.js >> data_$(date +\%Y_\%m_\%d_\%I_\%M_\%p).txt - -{{< /file >}} diff --git a/docs/guides/development/python/check-python-version/index.md b/docs/guides/development/python/check-python-version/index.md deleted file mode 100644 index 9c6b07f37db..00000000000 --- a/docs/guides/development/python/check-python-version/index.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -slug: check-python-version -title: "Check Your Python Version" -title_meta: "How to Check Which Python Version Is Installed" -description: "Knowing your Python version can make the difference between an application running or frustratingly failing. Thankfully, there is a quick command, and even some Python script, to check your currently installed Python version. Find out all you need to know about getting your Python version in this guide." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2022-08-15 -keywords: ['check python version','how to check python version','python version command'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[phoenixNAP: How to Check Python Version in Linux, Mac, & Windows](https://phoenixnap.com/kb/check-python-version)' -- '[LearnPython: How to Check Your Python Version](https://learnpython.com/blog/check-python-version/)' ---- - -Python reigns as one of the most popular programming languages, with a wide range of programs and developer tools relying on it. In fact, your system likely already has at least one version of Python installed. - -Many tools and Python development libraries require a particular version of Python. Thus, you may want to know where you can find information on your installed Python version. This can help you make decisions about compatibility, upgrades, and more. - -This tutorial shows you how to check your Python version, for both Python 2 and Python 3. Here, you can find the command line method as well as a Python script method for retrieving the current Python version. - -## How to Check the Python Version from the Command Line - -The Python command comes with a command line option of `--version` that allows you to see your installed version. - -It works just as straightforwardly as it sounds. Enter the following command from your command line, and you should get an output similar to the one shown below: - - python --version - -{{< output >}} -Python 3.8.10 -{{< /output >}} - -### Python 2 vs Python 3 - -Some systems distinguish between Python 2 and Python 3 installations. In these cases, to check your version of Python 3, you need to use the command `python3` instead of `python`. - -In fact, some systems use the `python3` command even when they do not have Python 2 installed alongside Python 3. In these cases, you only have the `python3` command. - -The command for checking the installed version of Python 3 remains otherwise the same - just use `python3` with the `--version` option: - - python3 --version - -## How to Check the Python Version from Python - -You can also check your installed Python version from within Python itself. Using either a script or the Python shell, you can use one of the code snippets below to print your Python version. - -Both options work equally well regardless of your system. The choice of which option to use really comes down to what format you want the output in. - -### Using sys - -The `sys` module has a variable you can reference to get the current Python version. Below you can see an example of how the `sys` module's `version` variable renders the current Python version. This code first imports the `sys` module then prints out the contents of the `version` variable: - -```python -import sys - -print(sys.version) -``` - -{{< output >}} -3.8.10 (default, Jun 22 2022, 20:18:18) -[GCC 9.4.0] -{{< /output >}} - -As you can see, the `sys.version` variable contains more information about your installed Python version than just the number. For that reason, `sys` is a good module to turn to when you want more verbose version information. - -### Using platform - -The `platform` module includes a function that fetches the current version of Python. The example code below uses this function to print the current Python version number. It first imports the `platform` module; then, the `python_version` function returns the version number to the `print` function: - -```python -import platform - -print(platform.python_version()) -``` - -{{< output >}} -3.8.10 -{{< /output >}} - -The output from the `platform.python_version` is more minimal compared to the `sys` module's `version` variable. This makes the `platform` module more useful for cases when you only need the version number. For example, this method helps when you want to design a program to parse the Python version and act accordingly. - -## Conclusion - -With that, you have everything you need for checking your current Python version. The steps above cover you whether you need to see the Python version from the command line or from within a Python script. - -You can continue learning about Python with our collection of [Python guides](/cloud/guides/development/python). We cover everything from fundamental Python concepts to building Python web applications. diff --git a/docs/guides/development/python/create-a-python-virtualenv-on-centos-8/index.md b/docs/guides/development/python/create-a-python-virtualenv-on-centos-8/index.md deleted file mode 100644 index 17bb5333bc5..00000000000 --- a/docs/guides/development/python/create-a-python-virtualenv-on-centos-8/index.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -slug: create-a-python-virtualenv-on-centos-8 -title: Creating a Python Virtual Environment on CentOS 8 -title_meta: How to Create a Python Virtual Environment on CentOS 8 -description: This guide provides a brief introduction to Python virtual environments using the virtualenv tool on CentOS 8. -authors: ["Linode"] -contributors: ["Linode"] -published: 2017-08-13 -modified: 2020-04-07 -keywords: ["python", "python virtual environment", "virtualenv", "centos 8"] -tags: ["python","centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[virtualenv Official Documentation](http://virtualenv.pypa.io/)' -audiences: ["beginner"] -languages: ["python"] -image: Python_virtualenv_CentOS8.png -relations: - platform: - key: python-virtual-env - keywords: - - distribution: CentOS 8 -aliases: [] ---- - -## What is a Python Virtual Environment? - -A Python virtual environment is an isolated project space on your system that contains its own Python executable, packages, and modules. Your Python applications and projects often have their own specific dependencies. With a virtual environment you can manage each of your project's distinct dependencies without having them interfere with each other. You can use the [*virtualenv*](https://pypi.org/project/virtualenv/) tool to create a virtual environment on your system. This guide will show you how to use virtualenv to create and run a Python virtual environment on a CentOS 8 Linode. - - -## Before You Begin - -1. Complete the [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guides to prepare your system. - -1. Update your system: - - sudo yum update - - {{< note respectIndent=false >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. - {{< /note >}} - -## Create a Python Virtual Environment -{{< note >}} -CentOS 8 does not include any version of Python by default. To install Python on CentOS 8, read our guide on [installing Python 3 on CentOS 8](/cloud/guides/how-to-install-python-on-centos-8) -{{< /note >}} - -1. To install Python's virtual environment: - - sudo yum install virtualenv - -1. Create a `python-environments` directory in your user's home directory and navigate to it: - - mkdir ~/python-environments && cd ~/python-environments - -1. Create a Python virtual environment. By default, virtualenv attempts to use your system's default Python interpreter to create a new environment. Replace `env` with the name you would like to assign to your virtual environment. - - virtualenv env - - {{< note respectIndent=false >}} -If your CentOS 8 system has another version of Python installed and you'd like to use it to create your virtual environment, use the e`--python` option to designate it. For example: - - virtualenv --python=python2.7 env - {{< /note >}} - -1. Validate that your environment is installed with the version of Python that you expect: - - ls env/lib - - You should see your `env` environments Python version: - - {{< output >}} -python3.6.8 - {{}} - -## Activate Your Virtual Environment - -1. Activate the newly created virtual environment: - - source env/bin/activate - - The name of the working environment appears in parentheses after it's created. - - {{< output >}} -(env) example_user@hostname:~/python-environments$ - {{}} - - You can now begin installing Python packages and libraries that will remain isolated to your virtual environment. - -## Deactivate a Virtual Environment - -1. To deactivate an active virtual environment, issue the following command: - - deactivate - - Your virtual environment is deactivated and you should no longer see its name listed next to your command line's prompt - - {{< output >}} -example_user@hostname:~/python-environments$ - {{}} diff --git a/docs/guides/development/python/how-to-use-unicode-in-python3/index.md b/docs/guides/development/python/how-to-use-unicode-in-python3/index.md deleted file mode 100644 index 5890541893a..00000000000 --- a/docs/guides/development/python/how-to-use-unicode-in-python3/index.md +++ /dev/null @@ -1,399 +0,0 @@ ---- -slug: how-to-use-unicode-in-python3 -title: "How to Use Unicode in Python 3" -title_meta: "Using Unicode in Python 3" -description: 'This guide introduces the concept of Unicode to developers, explains how Python handles unicode, and demonstrates how to handle common errors' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2023-03-20 -keywords: ['Python unicode','Unicode python','Unicode error python','Python unicode to string'] -tags: ['python'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Python Unicode Documentation](https://docs.python.org/3/howto/unicode.html)' -- '[Unicode site](https://home.unicode.org/)' -- '[Unicode 14.0 documentation](https://www.unicode.org/versions/Unicode14.0.0/)' -- '[Wikipedia ASCII page, including code points](https://en.wikipedia.org/wiki/ASCII)' -- '[Wikipedia UTF-8 page](https://en.wikipedia.org/wiki/UTF-8)' -- '[RFC 3629](https://datatracker.ietf.org/doc/html/rfc3629)' -- '[An explanation of the Unicode standard](https://jkorpela.fi/unicode/guide.html)' -- '[Python background on lexical analysis](https://docs.python.org/3/reference/lexical_analysis.html)' ---- - -Most of the time, using [*Unicode*](https://home.unicode.org/) characters in Python does not require extra effort. However, sometimes encoding and decoding do not work properly, which results in errors. To resolve these issues, this guide helps you understand how Python encodes and decodes Unicode. Fortunately, the Python library includes some powerful and useful utilities and built-in functions to manage these tasks. This guide introduces Unicode and the [*UTF-8*](https://en.wikipedia.org/wiki/UTF-8) character encoding and explains how Python handles Unicode. It also discusses some common Python Unicode errors and demonstrates how to resolve them. - -For an in-depth explanation of Unicode, read on, otherwise jump to [How Does Python Implement Unicode?](#how-does-python-implement-unicode) - -## An Introduction to Unicode on Python - -To properly understand how Python manages Unicode, you need to understand character processing. Computer files are written using a specific character set. A character set is a collection of characters used within a language or domain. For instance, the written English language maps to a character set containing 26 upper and lower case letters, along with punctuation marks. However, computers use a more formal collection called a *Coded Character Set* (CCS). - -In a CCS, each character is assigned a corresponding numerical value known as a *code point*. A set of code points makes it possible for a system to translate between the on-screen representation of each character and its binary equivalent. Every character set is also associated with a *code unit*. The code unit determines the size of each encoded character. For instance, a character set could encode every single character using 16 bits, or two bytes. The same code point could be associated with different code units in different encodings. For instance, the value `127` might be represented using 7, 8, 16, or 32 bits. Some formats even use a variable-length encoding. This means it is not possible to determine the binary representation of a character based on its code point. - -Text is decoded and encoded using a specific encoding standard. The encoding algorithm is known as a *codec*, which is a portmanteau of the coder/decoder. The system must know what codec to use to encode/decode the file correctly. Files can be decoded from bytes to characters and text can be encoded from characters to bytes. Format conversions can certainly become complicated, but most files use either the ASCII or Unicode format. - -## What is Unicode? - -Unicode is currently the most widely-used encoding standard. It was developed through the joint efforts of Xerox and Apple, but it is currently administered and maintained by the Unicode Consortium. Unicode has successfully unified pre-existing character sets and now serves as the international standard. Unicode has the goal of including every character used in the world's active writing systems. - -Unicode describes the list of available characters and their code points but does not describe how to map the code points to bytes. It also includes a formal Unicode name for each character. A variety of *character encoding schemes* (CES) can be applied to Unicode text to map the characters to bytes. - -The Unicode standard includes the following components: - -- **Character Repertoire**: This is the full set of characters that Unicode supports. Unicode currently defines nearly 150,000 characters. The repertoire is open to new additions, and additional characters are always being proposed. The characters are subdivided into several sub-components known as general categories. Some examples of general categories are `letter`, `mark`, `number`, and `symbol`. -- **Coded Character Set (CCS)**: Like other CCS systems, the Unicode standard maps each character to a code point. Each code point represents one and only one character. For instance, Unicode hexadecimal code point `1F6A6` represents the traffic light emoji 🚦. The Unicode code point for a given character can differ from the code points used in other systems, although all ASCII characters continue to use the ASCII code points. -- **Character Encoding Form (CEF)**: This component explains how to map code points to code units. -- **Character Encoding Scheme (CES)**: The encoding scheme maps code units to a sequence of bytes. It describes how to transmit files over a network and how to store character information in binary format. Unicode does not recommend a specific CES. Standards including UTF-8, UTF-32, or ISO/IEC 2022 can be used. However, UTF-8 is usually favored, mainly because it is more concise than the other systems. - -The Unicode character set includes all traditional ASCII characters, international writing scripts, symbols, and a large number of emojis. Unicode also contains various control and non-printable characters. Each Unicode letter is identified using the letter `U`, the `+` sign, and its code point. For example, the Unicode character having code point `639` is represented by the string `U+0639`. - -The Unicode code space is divided into seventeen planes to help structure and organize the collection. Related characters are placed within contiguous blocks inside a single plane. This makes it easier to locate specific characters in the published Unicode charts. All characters have a fixed name that uniquely identifies them. This name cannot be subsequently changed, even if it is inaccurate or contains errors. - -Unicode contains several unallocated "non-character" code points and blocks of private-use code points. The private characters can be used internally or through an agreement between a sender and receiver. In addition, a set of "formatting" characters modifies the behavior of adjacent characters, including ligatures. Some abstract characters can only be represented by a sequence of two or more characters. Unicode maintains a list of these abstract characters, but they are often the source of some confusion. - -Although moderated, Unicode is an open system. New additions are always being proposed. Unicode can expand to over one million code points, and there is still plenty of room for new characters. The current release of Unicode is 14.0, which was released in 2021. - -Most operating systems, web browsers, text processors, and programming languages such as Python have built-in support for Unicode. They can decipher Unicode-encoded text and display the appropriate characters. Some applications do not support Unicode or have not yet implemented the latest release. In this case, users might see empty rectangles or the `?` symbol in place of actual text. Some systems only support one or two-byte Unicode characters, which is a subset of the entire collection. - -The [Unicode site](https://home.unicode.org/) provides a complete overview of the Unicode standard, along with a FAQ and explanation of how to use the site. Perhaps the most important section of the site is the [specification of the latest release](https://www.unicode.org/versions/Unicode14.0.0/). The documentation allows developers to view the latest code charts and read an overview of each section. - -### What is the Difference between ASCII and Unicode? - -ASCII is a very early character encoding developed from telegraph code. It can be considered an ancestor of the current Unicode system. ASCII is an abbreviation of the American Standard Code for Information Interchange. It was the standard during the early days of the internet until approximately 2008 and is still common today. - -The ASCII character set consists of 128 characters, which have code points between 0 and 127. Each code point is represented as a 7-bit binary number. Some applications used the eighth and final bit for proprietary purposes, leading to a lack of compatibility. The set of ASCII characters was found to be too small and restrictive, so new encodings were developed to allow for more characters. However, most modern encoding systems are based on ASCII and typically preserve the original code points for the original ASCII characters. - -Within the ASCII character set, control characters and related characters, such as digits, have contiguous code points. This makes it easier to locate characters. The ASCII system consists of the following characters. - -{{< note >}} -All code points in this list use the decimal system. -{{< /note >}} - -- Lowercase letters from `a` to `z`. These have code points between `97` to `122`. -- Uppercase letters from `A` to `Z`. They are assigned code points between `65` to `90`. -- The digits `0` to `9`, run from code points `30` to `39`. -- The space character, with code point `32`. -- Essential punctuation points. -- Opening and closing brackets, parentheses, braces, chevrons, and forward/backward slashes. -- A small set of mathematical, typographical, and other symbols, including `+`, `&`, `$`, and `^`. -- A set of 32 non-printable control characters, including codes for a line return, tab, bell, backspace, and form feed. These characters have code points `1` to `31`, and `127`. Many of these codes apply to archaic printing devices and are no longer in use. A few of these codes have been assigned new meanings. -- A `null` character that has code point `0`. - -A complete list of the characters can be found on the [Wikipedia ASCII page](https://en.wikipedia.org/wiki/ASCII#Printable_characters). The ordering of characters based on their ASCII code points is known as *ASCIIbetical order*. Computer systems sometimes process characters using ASCIIbetical order rather than alphabetical order. - -By contrast, Unicode is an expanded and updated encoding standard that builds upon the original ASCII standard. Unicode separates the code points from the details of the encoding system. This permits a much wider range of characters up to four bytes. - -The Unicode character set incorporates the entirety of the ASCII character set as the first 127 characters. All ASCII characters have the same code points in both encodings. This means any ASCII text file is a valid UTF-8 file. Additionally, any UTF-8 file that only uses ASCII characters can be processed as ASCII text. This ensures legacy ASCII-only applications can accept files with only ASCII characters, maintaining backward compatibility. - -The greater flexibility of Unicode allowed it to become the modern standard for character encoding in computer systems and on the internet. However, ASCII-only applications and systems persist to this day. ASCII-only applications and files are guaranteed to be compatible with any system. - -### What is UTF-8? - -UTF-8 is one of several character encoding schemes implementing the Unicode encoding standard. An encoding scheme translates a string to a byte sequence and a byte sequence back into a string. UTF-8 is defined in [RFC 3629](https://datatracker.ietf.org/doc/html/rfc3629). It is capable of encoding all 1,112,064 code points in Unicode. - -UTF-8 is backward-compatible with ASCII. All ASCII files are UTF-8 compliant and a UTF-8 file that only contains ASCII characters is compatible with ASCII systems. - -A code unit in UTF-8 is 8 bits, compared to 7 bits for ASCII. However, one character can map to anywhere between one and four code units. The alternative UTF-32 codec represents all characters using four code units each consisting of four bytes. This means UTF-8 generates much smaller files than UTF-32. - -UTF-8 is the most popular encoding on the internet and is used in between 95 to 100% of all websites. It is now considered the international standard for Unicode encoding. Virtually all modern applications support UTF-8, and many standards and applications only accept files encoded in UTF-8. UTF-8 files are safe to use with programming languages that use escape or special characters. - -Here are some of the main advantages of UTF-8: - -- UTF-8 can encode all possible Unicode code points. -- It only uses a byte containing all zeroes to represent the `null` character. This maximizes interoperability with the C programming language and legacy protocols that use `0` to indicate the end of a file. -- All ASCII strings are valid UTF-8 strings and have the same code points. This means UTF-8 can encode ASCII files. Seven-bit values never otherwise occur in UTF-8, avoiding confusion. -- UTF-8 is a compact and concise encoding. Most Unicode characters can be stored in one or two bytes. -- UTF-8 can resynchronize at the next code point to recover from corrupted or lost data. Because certain values cannot occur in UTF-8, errors are easier to detect. The start of a character can be found within three bytes of any point in the file, and the sequence for one character never begins inside another character. -- It avoids the bytes `FF` or `FE`, to avoid confusion with the UTF-16 byte order mark. This means it is impossible to misinterpret a UTF-8 file as UTF-16. -- It is byte-oriented and the encoding does not depend on the underlying hardware architecture. The first byte indicates the number of bytes of the next character. -- If a file gets split in the middle of a character, only a single character is lost. - -The Unicode encoding process follows an establishing encoding formula, which designates certain bits as framing bits. In practice, this restricts the maximum value to 21 bits. The encoding technique can be described as follows. - -- One-byte values begin with a `0`, followed by the 7 bits of the code point. All ASCII values, and only the ASCII values, are encoded as one-byte values. -- For two-byte values, the first byte begins with `110`, and the second with `10`. A byte beginning with `10` is a *continuation byte*. The remainder of the bits encodes the code point. These values are used for all Latin-style alphabets, along with Greek, Cyrillic, Hebrew, Arabic, and some other languages. Two-byte values also encode diacritical marks and International Phonetic Alphabet (IPA) extensions. -- Three-byte values start with `1110`. The second and third bytes begin with `10`. This plane of values encodes the Basic Multilingual Plane, including almost all characters commonly in use, and many emojis. -- Four-byte values are indicated with leading `11110` bits. The remaining three bytes start with `10`. Emojis, mathematical symbols, and some ideographic symbols from Asian languages are encoded using four bytes. -- Certain bit patterns indicate an invalid sequence. This could indicate a potentially valid code point that has not been assigned a value yet or an invalid UTF-8 value. An example is an unexpected continuation byte. Most invalid values are dropped, but hackers have used them to bypass security measures. -- The byte order mark `0xEF 0xBB 0xBF` at the start of a file indicates the file uses UTF-8. However, it is not recommended or required and might confuse some systems. If the file begins with the byte order mark `FEFF`, the file is encoded in UTF-16, not UTF-8. - -As an example, here is how to encode the Unicode code point for the Peso symbol `₱`. - -- The Unicode code point for this symbol is `0x20B1`. It can be found in the currency chart in the [Unicode charts](https://www.unicode.org/charts/). -- This value requires a three-byte encoding, with 16 bits used for the code point and eight bits for the framing. -- Converted to binary, this value is `0010 0000 1011 0001`. -- The first four bits are encoded in the first byte and preceded by the framing bits `1110`. This results in `1110 0010`. -- The second byte begins with `10`, followed by the next six bits of the code point. This results in `1000 0010`. -- The third and final byte also starts with `10`, followed by the remaining encoding bits. This yields `1011 0001`. -- Taken together, this is `1110 0010 1000 0010 1011 0001`. In hexadecimal, this is `0xE282B1`. - -UTF-8 can be contrasted with the UTF-16 and UTF-32 encoding schemes. UTF-16 always uses at least two bytes to encode characters. It can also use four bytes, while UTF-32 always uses four bytes. These schemes are somewhat easier to decode and use fewer framing bits. However, they are less concise than UTF-8 and are not ASCII-compatible. - -## How Does Python Implement Unicode? - -Python handles Unicode very differently in Python 2 and Python 3. In Python 2, the default encoding is ASCII. But in Python 3, UTF-8 is the default. This makes text processing much easier than before. It is no longer necessary to declare the encoding in the first or second line of the file. Nor is it necessary to preface Unicode strings with `u`. - -{{< note >}} -This guide only discusses Unicode processing for Python 3. Python 2 handles Unicode very differently, so these instructions do not apply. For more information, consult the [Python 2 Unicode documentation](https://docs.python.org/2/howto/unicode.html) for further guidance. Python strongly recommends that developers upgrade to Python 3. -{{< /note >}} - -According to the Python [Unicode documentation](https://docs.python.org/3/howto/unicode.html), the most important principle is to only work with Unicode strings internally. Decode the input data as soon as possible and encode the output only at the end. This greatly simplifies most programs and avoids introducing errors. - -All Python 3 strings are Unicode strings and are stored as Unicode. This means Unicode characters can be included in a string. When the text is encoded, all characters are converted to their byte equivalent. To use Unicode characters in a string, declare the string normally, and include the Unicode characters in the correct position. - -```file -u = "❤️" -print(u) -``` - -```output -❤️ -``` - -Python 3 also permits many Unicode characters to be used as part of variable and function names. However, symbols or emojis can't serve in these roles. For instance, the "heart" emoji can not be part of a variable name. In the following example, the variable name `Øyen` contains a character from the Norwegian alphabet. - -```file -Øyen = "Lofoten" -print(Øyen) -``` - -```output -Lofoten -``` - -A string can still be declared using a `u` at the front of the string, but this is no longer required. If a developer is simultaneously processing Unicode and ASCII files, they might want to use this convention to increase clarity. This is also necessary to backport files to Python 2. - -```file -u = u"❤️" -``` - -A character can also be declared using its *escape sequence*. The Unicode escape sequence for two-byte characters is a backslash `\` character, a `u`, and four hexadecimal digits indicating the code point. If the hexadecimal character is shorter than four digits, insert leading zeros to pad the length to four. - -The Unicode code point for the musical note symbol `♩` is `0x2669`. This symbol can be assigned to a Python string using its hexadecimal equivalent. - -```file -x = "\u2669" -print(x) -``` - -```output -♩ -``` - -The escape sequence for a Unicode character requiring three or four bytes begins with `\U`. The hexadecimal value must contain eight digits, so pad the front of the value with zeros until it is the proper length. - -For example, the Unicode bumblebee emoji is encoded in a three-byte format possessing the Unicode code point `U+1F41D`. To assign this emoji character to a variable using its escape sequence, pad it out to `0001F41D`. After the character is assigned to a string, it can be printed out using the Python `print` function. - -```file -bee = "\U0001F41D" -print(bee) -``` - -```output -🐝 -``` - -### Using the Python `unicodedata` Library - -Python provides many additional functions and libraries to help developers work with Unicode. The most relevant library is `unicodedata`. It allows developers to extract more information, such as the official name or code point, about each Unicode character. This library can be imported using the following Python directive. - -```file -import unicodedata -``` - -To find the code point of a Unicode character, use the `ord` function. Precede the character with a `u`. This example demonstrates how to determine the code point for the musical note symbol. - -{{< note >}} -This function only returns valid results for Unicode characters with a single code point. Although `ord` is a core Python function, the `unicodedata` library is necessary to determine multi-byte mappings. -{{< /note >}} - -```file -print(ord(u"♩")) -``` - -```output -9833 -``` - -The opposite function of `ord` is `chr`. It is used to convert the decimal integer of a code point to the actual character. - -```file -print(chr(9833)) -``` - -```output -♩ -``` - -The `name` method is used to retrieve the official Unicode name for any Unicode character. - -```file -print(unicodedata.name(u"🐝")) -``` - -```output -HONEYBEE -``` - -The official Unicode name can also be used to determine the character. Pass the official name, including all spaces, to the `lookup` method. - -```file -print(unicodedata.lookup("QUARTER NOTE")) -``` - -```output -♩ -``` - -A character can also be assigned to a string using its official Unicode name. Enclose the name in braces and precede it with a backslash and `N`. The following example assigns the musical note symbol to the `note` string. - -```file -note = "\N{QUARTER NOTE}" -print(note) -``` - -```output -♩ -``` - -### Reading and Writing Unicode Files in Python - -The same concepts are used to read a file containing Unicode characters. Declare the file encoding as `utf-8` when calling the `open` function. Read the file normally. - -```file {title="read_unicode.py"} -with open('sample.txt', encoding='utf-8') as f: - for line in f: - print(repr(line)) -``` - -Given a `sample.txt` file containing some of the Unicode characters used in this guide, `read_unicode.py` returns the following results. - -```output -'Øyen\n' -'♩\n' -'❤️\n' -'🐝\n' -``` - -For a write operation, create the file object as follows. Ensure the encoding is set to `utf-8`. The `write` method can accept Unicode characters or code points preceded by `\u` or `\U`. - -```file {title="write_unicode.py"} -with open('sample.txt', encoding='utf-8', mode='w') as f: -``` - -For more information on reading and writing Unicode files, consult the [Python Unicode documentation](https://docs.python.org/3/howto/unicode.html). - -### Limitations - -Python 3 has far fewer limitations than Python 2. However, certain functions might not work as expected for characters requiring more than one code point or interactions between two consecutive characters. A good example of this is the `ord` function, which expects a single code point. - -Always confirm the encoding of an input file unless it is known to be a UTF-8 file. Usually, the file specifies the encoding in the first or second line. However, sometimes the codec is not clear, so developers might have to contact the file originator. - -### Encoding Text in Python - -Encoding and decoding is the process of converting from strings to bytes. Strings display the text in a human-readable format, and bytes store the characters as binary data. Encoding converts data from a character string to a series of bytes. Decoding translates the bytes back to human-readable characters and symbols. It is important not to confuse these two methods. `encode` is a string method, while `decode` is a method of the Python byte object. It is not possible to encode bytes because the bytes are already encoded. Likewise, it is not possible to decode a string. A string is already a series of characters and it technically does not have any encoding. The string object does not even have a `decode` method. Attempting to call `decode` for a string results in an error. - -To use Python to encode Unicode characters, use the string `encode` method. The default encoding standard is UTF-8, but for reasons of clarity, it is good practice to always explicitly pass in the protocol. The following example demonstrates how to encode the musical note Unicode character. Python converts the Unicode string to bytes and returns an encoded byte object. - -```file -note = "♩" -note.encode("utf-8") -``` - -```output -b'\xe2\x99\xa9' -``` - -The `encode` method works the same way on multi-character strings. The ASCII characters are not converted to bytes. This preserves backward compatibility with ASCII-only applications. - -```file -greeting = "Have a nice day! ♩" -greeting.encode("utf-8") -``` - -```output -b'Have a nice day! \xe2\x99\xa9' -``` - -### Decoding Unicode Bytes in Python - -The inverse function to `encode` is `decode`. If this function is applied to a sequence of bytes, it returns the equivalent Unicode string of characters. To decode a function, use the `decode` method of the byte object. The encoding protocol is assumed to be UTF-8 by default, but it is safer to explicitly state it. This example decodes the byte encoding for the note character back into a string. - -```file -notebytes = b'\xe2\x99\xa9' -notebytes.decode("utf-8") -``` - -```output -'♩' -``` - -## Common Unicode Errors in Python - -Most of the time, the encoding and decoding process goes smoothly without much extra effort. However, Unicode errors can occur in Python. When Python cannot decode a file, it displays the `UnicodeDecodeError` message. - -The most common Python Unicode error happens when a non-UTF-8 file is decoded using the UTF-8 codec. In this example, a string is encoded using the `latin-1` encoding. Attempting to decode it as a UTF-8 file results in a `UnicodeDecodeError`. - -```file -quartercup = "¼" -quart_encode = quartercup.encode("latin-1") -quart_encode.decode("utf-8") -``` - -```output -UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbc in position 0: invalid start byte -``` - -To decode this string properly, declare it as a `latin-1` file in the function call. The type of encoding is often specified in the first line of the file. Python searches for the keywords `coding: name` or `coding=name`. But the convention is to use the format `# -*- coding: latin-1 -*-`, a carryover from the emacs editor. - -```file -quart_encode.decode("latin-1") -``` - -```output -'¼' -``` - -This error can also occur when the `decode` method is applied to a byte object containing invalid Unicode sequences. This can happen if the file was corrupted or the characters were not encoded correctly. In this example, the `0xfe` byte is assigned to a Python byte object. This character is not permitted anywhere in a UTF-8 file because `xfe` is reserved for the UTF-16 byte order mark. If this byte is decoded using the UTF-8 decoder, Python raises an error. - -```file -junk_bytes = b'\xfe' -junk_bytes.decode("utf-8") -``` - -```output -UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 0: invalid start byte -``` - -The `UnicodeEncodeError` is less common because developers generally have some control over the material they are decoding. However, it can happen if a developer attempts to encode Unicode data using another encoding algorithm. In this case, encoding is used on a string containing the bumblebee emoji. The string can be correctly converted to bytes using the UTF-8 encoder. However, when `encode` is invoked with the `ASCII` encoder, the attempt fails, generating the `UnicodeEncodeError`. This is because the bumblebee emoji does not exist in ASCII, and the ASCII codec cannot translate the Unicode code point. - -```file -bee = "\U0001F41D" -utf8_bee = bee.encode("utf-8") -ascii_bee = bee.encode("ascii") -``` - -```output -UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f41d' in position 0: ordinal not in range(128) -``` - -{{< note >}} -This UTF-8 codec permits unassigned Unicode code points to be encoded to bytes. However, these byte objects cannot later be decoded back into characters. -{{< /note >}} - -To avoid errors, `encode` and `decode` accept the `replace` keyword as the second parameter. This substitutes the Unicode replacement character, `U+FFFD` for any character that cannot be properly translated. The `ignore` keyword completely ignores unknown characters. However, most developers prefer to avoid silent errors and receive an indication something has gone wrong. - -```file -junk_bytes = b'\xfe' -junk_bytes.decode("utf-8", "replace") -``` - -```output -'�' -``` - -## Concluding Thoughts About Unicode on Python - -Unicode was designed as a replacement and extension for the ASCII character set, which only contained 128 characters. Unicode covers characters from almost every world language, along with many symbols and emoji. Each character has an equivalent code point, which is used to identify, display, and translate the character. - -UTF-8 is an encoding technique for converting Unicode characters to bytes and vice versa. It uses between one and four bytes to store a Unicode code point and is fully backward-compatible with ASCII. Python fully supports both Unicode and UTF-8 and permits strings to include any Unicode character. It includes the `unicodedata` library, which allows Python to manipulate Unicode data. Python decodes and encodes Unicode data using built-in string and byte methods. For more information on using Unicode in Python, see the [Python documentation](https://docs.python.org/3/howto/unicode.html). diff --git a/docs/guides/development/python/pros-and-cons-of-python/index.md b/docs/guides/development/python/pros-and-cons-of-python/index.md deleted file mode 100644 index 36cd0afd9ec..00000000000 --- a/docs/guides/development/python/pros-and-cons-of-python/index.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -slug: pros-and-cons-of-python -title: "The Pros and Cons of Python Programming" -title_meta: "A Programmers’ Guide to Python: Advantages & Disadvantages" -description: 'As with any programming language, there are pros and cons of Python. Read our guide to find out whether you can benefit from learning Python. Click here!' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2022-03-23 -keywords: ['benefits of python','pros and cons of python','python advantages','disadvantages of python'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Python.org web site](https://www.python.org/)' -- '[Python documentation](https://docs.python.org/3/contents.html)' -- '[Python Package Index (PyPI) repository](https://pypi.org/)' -- '[Python Beginner Guide](https://wiki.python.org/moin/BeginnersGuide)' -- '[Python downloads page](https://www.python.org/downloads/)' -- '[Django](https://www.djangoproject.com/)' -- '[Flask](https://flask.palletsprojects.com/en/2.0.x/)' -- '[TensorFlow](https://www.tensorflow.org/)' -- '[PyTorch](https://pytorch.org/)' -- '[NumPy](https://numpy.org/)' -- '[SciPy](https://scipy.org/)' -- '[Python licensing site](https://docs.python.org/3/license.html)' ---- - -[Python](https://www.python.org/) is currently one of the world's most popular programming languages. It achieved this position due to its powerful features, flexibility, and ease of use. This guide explains the pros and cons of Python and compares it to other languages. It also discusses the situations where Python is one of the best alternatives. - -## What is Python? - -### The History of Python - -Python is an open source, general purpose programming language. Guido van Rossum developed Python based on the defunct ABC programming language and named it after the Monty Python comedy troupe. Python was designed to be simple, readable, and highly extensible through the use of modules. The first version of Python was released in 1991, and the more fully featured Python 2.0 followed in 2000. Both releases have since been discontinued. - -Python 3.0 was introduced in 2008, but is not compatible with earlier releases. Even though Python included the `2to3` upgrade utility, this decision was highly controversial and created significant issues for the user base. Because the transition was so difficult, there is unlikely to be a release 4 of Python. Instead, new minor releases are planned for each year. The Python Software Foundation currently administers Python, and they continue to work on new features and ongoing performance improvements. - -Python continues to increase in popularity and is now ranked as one of the top five languages. It is widely used in data science, machine learning, artificial intelligence, and server & web applications. Many web developers use Python alongside external frameworks, including [Django](https://www.djangoproject.com/) and [Flask](https://flask.palletsprojects.com/en/2.0.x/), or third-party libraries. These frameworks include ready-to-use components and are especially useful for web development. - -### The Main Characteristics of Python - -Python uses many of the same concepts, commands, and control structures as other traditional programming languages. But it is different in many respects, and can almost be considered a new programming paradigm. Python promotes flexibility and clear coding design, and code adhering to its principles is said to be "Pythonic". Here are some of Python's main characteristics. - -- **Python is a High-Level Language**: High-level languages are more readable. They use meaningful variable names, and have meaningful syntax. No understanding of the underlying operating system is required. In this respect, Python is similar to other programming languages, including JavaScript, Rust, and C++, but is even more clear and legible. At the opposite end of the spectrum is assembly language. Assembly code refers to memory addresses and uses machine language instructions. -- **Python Supports Object-Oriented Programming (OOP)**: Python is an OOP language with support for classes, methods, inheritance, and encapsulation. Unlike Java, Python does not enforce an OOP model and object-oriented design principles are strictly optional. So it is possible to use Python strictly in imperative/procedural mode for short programs and simple utilities. Python now incorporates some features from the functional programming paradigm, but it is not considered a true functional programming language. -- **Python is a General-Purpose Language**: Domain-specific languages are intended for one specific purpose. For example, SQL is only used to communicate with relational database systems. However, Python is a general-purpose language, and has a wide range of applications. -- **Python is an Interpreted Language**: Unlike many languages, developers do not have to compile Python into assembly or machine code. When a developer completes a program, they can immediately run it with no intermediate steps. The Python interpreter deciphers each line at run time and executes it. This is different from languages like C/C++, which must be pre-compiled first. Python does have a compilation stage, but it takes place at run time and is hidden from the user. Python compiles a program down to low-level *bytecode* for the *Python Virtual Machine* (PVM) to interpret and execute. -- **Python is Dynamically-Typed**: Variables do not have to be assigned a type, such as "integer", when they are first used. Python determines the type of variable at run time. Python uses a technique known as "duck typing". It assigns a type to a variable depending on its value and how it is used. Python allows a variable to change type dynamically over the duration of the program. -- **Python Programs are Platform-Independent**: Because Python programs are interpreted, they can be ported to any platform. Only the Python Virtual Machine is platform-specific. It translates the Python code into valid machine code for the platform it is running on. - -## Pros and Cons of Python - -Python is a very distinctive language that has both pros and cons. It is great for certain situations and not as good for others. This section highlights both the advantages and disadvantages of Python. - -### Advantages of Python - -Python has become widely used and well liked due to a cluster of positive attributes. Here are some of the benefits of Python. - -- **Ease of Use**: Python has a simple, concise, and straightforward syntax. A Python program looks a lot like plain English and is highly readable. This makes Python programs easy to read and debug. Python's control structures are intuitive and easy to use. In addition, Python is dynamically typed, so there is no requirement to declare the type of each variable. For these reasons, Python is one of the most efficient and productive languages. -- **Gentle Learning Curve**: Python is one of the simpler languages to learn and is a good option for people learning to program. Programmers switching to Python from languages like C or Java can quickly reach peak efficiency. The Python package contains a useful *Integrated Development and Learning Environment* (IDLE). -- **Versatility**: Python is a flexible, general purpose language that fully supports both procedural and object-oriented programming. Due to its built-in and third-party packages, it is suitable for a wide range of tasks. It is dominant in the areas of data science and machine learning. It is also widely used for back-end web development and the *Internet of Things* (IoT). Even when it is not the best choice for a particular task, it is usually still a viable option. In addition, Python code can be embedded into projects written in other languages, such as C++, and code from other languages can be embedded in Python. -- **Efficient for Rapid Development**: Because Python is easy to use and does not have to be compiled, programs take less time to develop. Python programs are typically much shorter than equivalent programs in other languages. It is a great choice for quickly constructing prototypes in a rapid software development environment. -- **True Portability**: A huge Python advantage is that it can be written once and run anywhere. Python does not have to be compiled in advance, so users run a true Python program and not a Python executable. The program is not compiled until it is run, using the platform-specific PVM. This means any Python program can potentially run on any system that supports Python. -- **No Compile Process**: Python is an interpreted language and programs are automatically compiled at run time. A program can be run as soon as it is written. There is no separate compiler, no time-consuming compilation step, and no opaque compiler errors. Python programs are easy to write, debug, and change incrementally. -- **Automatic Memory Allocation**: Python does not have pointers and developers do not have to assign free space in memory. Python allocates memory automatically and a garbage collector recycles memory from discarded objects. This means developers do not have to worry about scribblers, memory leaks, invalid pointer references, or the size of each object. -- **Extensive Built-In Objects and Libraries**: Python has a large number of built-in compound objects including lists, sets, and record-like dictionaries. Each of these objects provides a collection of methods allowing for easy processing. In addition, Python has an extensive library containing tens of thousands of functions. These packages can be used for network communications, web integration, data processing, and hardware interactions. This makes it much faster to write programs because so many of the necessary routines have already been written. -- **Third-Party Library Availability**: In addition to Python's extensive built-in library, developers can access many free external libraries. These third-party libraries are easy to import and install using Python's `pip` package manager. Packages can be downloaded from the [Python Package Index (PyPI) repository](https://pypi.org/). PyPI also allows developers to publish their own packages. -- **Open Source and Free to Use**: All Python releases are available for free under an open source license. Python can even be modified and re-distributed at no cost. This greatly reduces development costs. For more information about Python licensing, see the [Python documentation site](https://docs.python.org/3/license.html). -- **Large User Base**: Python has a large, active, and passionate community of users. It is easy to find learning materials and other resources, ask questions, search for jobs, hire additional developers, and meet other Python programmers. - -### Disadvantages of Python - -Despite its many advantages, Python also has a few notable disadvantages. Here are some of the drawbacks of Python. - -- **Not Very Fast**: Python is much slower than more efficient languages like C and Java. Python is interpreted and dynamically-typed, so the run-time compiler has a lot of work to do. It must constantly validate the type of each variable. This means Python is not the best choice for scenarios where speed is critical. -- **Memory Intensive**: Python is not optimized to reduce memory. It can use ten times the RAM as a program written in a more frugal language. However, this is partly a tradeoff in return for flexibility and ease of use. In addition, the Python garbage collector cannot gather all discarded resources immediately, which reduces the amount of available memory. Python is not a good choice for memory-constrained environments. -- **Harder to Avoid Runtime Errors**: Python is not compiled until runtime and is dynamically typed. Therefore, many problems that would otherwise be caught by the compiler do not appear until the program runs. This might include something as simple as a syntax error, but it can include problems like trying to add an integer and a string together. -- **Not Much Traction in Mobile or Desktop Applications**: Because it is somewhat slow and uses a lot of memory, Python has not made gains in the mobile space. There are some Python development tools for mobile apps, but they are more limited than frameworks for other languages. The situation is a bit better in client desktops, but Python is still not too popular for front-end applications. -- **Not Optimized for Database Access**: It is more difficult to work with databases in Python than in some other applications. Python lacks a powerful, high quality, easy-to-use interface like the Java Database Connectivity (JDBC). It can still be used if the database reads and writes are relatively straightforward. But it is not the best choice for applications that have complex interactions with a large corporate database. -- **No Multithreading Support**: Due to its architecture, Python does not support multi-threading. Instead, it uses multiprocessing, where each "thread" runs in a separate Python process. This relies on the oversight of the operating system to schedule and balance the processes, and might not deliver equally good results. -- **Prone to Overuse or Misuse**: Python's simplicity is one of its strengths, but this can be a surprising weakness in some situations. Because it is so easy to use, it is often misused for tasks where it is not one of the best alternatives. Python is great for rapid development and prototypes, but this might tempt organizations to overlook proper software development principles. - -## Should You Learn Python? - -After reading about the pros and cons of Python, you might still be uncertain whether it is worth learning. On one hand, there are always benefits to learning a new language. But there are at least a dozen other popular languages, so there are also opportunity costs. - -Nonetheless, there are some situations where Python is the right choice. There are benefits to learning Python if any of the following statements apply. - -- **You are Using Agile Development**: Python is simple and concise, and it can be used to quickly work out a proof of concept or a trial prototype. Python's extensive library reduces the number of helper functions to write. It is easy to be fast and productive in Python. -- **You Work in Data Science or AI**: Python is especially strong in the areas of data science and machine learning. There are many external libraries available for these areas and a strong user base. For machine learning or natural language processing, [TensorFlow](https://www.tensorflow.org/) and [PyTorch](https://pytorch.org/) can be used. [NumPy](https://numpy.org/) and [SciPy](https://scipy.org/) are frequently used for data science or scientific computing. -- **You Require Portable Tools**: Python is a sensible choice for multi-platform utilities. Python is available for Linux, Windows, MacOS, and other popular platforms. It is compiled at run time, so the program can run anywhere on an interpreter optimized for the underlying operating system. -- **You Require Test Automation or DevOps Scripts**: Python is great for scripting because it is so quick and easy to write, modify, and debug. Speed and memory use are typically not of much concern in a QA environment, so there are no real downsides. Python has a built-in test library named [PyUnit](https://wiki.python.org/moin/PyUnit). PyUnit integrates smoothly with most test frameworks including `pytest`. On the DevOps side, Python forms the backbone of many configuration management programs. -- **You Require a Back-End for Web Applications**: Along with the popular Flask or Django frameworks, Python can provide the server-side engine for web applications. The frameworks provide libraries and templates for most web design tasks. -- **You Want a Free and Open Source Language**: Python is free to download, use, and modify under a versatile open source license. Python reduces development costs and makes it easy to upgrade to new releases. -- **You Want a Fun and Easy to Learn Language**: Python has an easy and straightforward syntax. It is a great choice for beginners who are learning how to program. It is also advantageous for professionals who want to dive into development without worrying about fussy programming details. Python's extensive libraries allow developers to solve problems at a high-level without writing complex low-level routines. -- **You Want a Large Developer Community**: Python is one of the most popular general-purpose languages. It is versatile and used in a wide number of specialties, so companies are always looking for developers. Google, Facebook, Microsoft, Spotify, Instagram, and Uber all use Python for at least some applications. Python is also a popular development language in start-ups. - -It is easy to start using Python. A good place to begin is the [Python Beginner's Guide](https://wiki.python.org/moin/BeginnersGuide). The Python Wiki also has a list of tutorials and resources for [New](https://wiki.python.org/moin/BeginnersGuide/NonProgrammers) or [Experienced Programmers](https://wiki.python.org/moin/BeginnersGuide/Programmers). - -To run Python on Ubuntu or another Linux distribution, use the command `python3`. Python is usually already installed on most Linux systems. To download Python for other platforms, see the [Python downloads page](https://www.python.org/downloads/). For more information about how to use Python on a Linode system, see the [Linode guide to Python](/cloud/guides/how-to-install-python-on-ubuntu-20-04). - -### Alternatives to Python - -For situations where Python is not the best choice, consider the following options. - -- **C/C++**: These two languages are strong choices when speed, performance, and low memory use are important. They are statically typed and require pre-compilation, so they generate fewer runtime errors. They are often used in game development and embedded systems. Unfortunately, they are considered difficult languages to master. -- **JavaScript**: Like Python, JavaScript is an interpreted and dynamically-typed language. However, it is the most common language for interactive web pages. JavaScript runs on the client side, and interacts seamlessly with HTML and CSS components. It can be used alongside Node.js for full stack web development. Python is not typically used on the front end, so it is not a good full stack alternative. -- **R**: R is an alternative to Python for data science. However, it is more geared toward statistical analysis. R is particularly good for data visualization, but it is more complicated and difficult to learn. Python is a simpler and faster option for general purpose data and numerical analysis and has better machine learning capabilities. - -## A Summary of the Pros and Cons of Python - -This guide discusses the pros and cons of Python, which is more effective in some situations than others. Python is an interpreted, statically-typed programming language that allows both object-oriented and procedural programming. Some of the main benefits of Python include its ease of use, concise and straightforward syntax, and vast libraries. Other Python advantages are its portability, versatility, large user base, and free & open source license. - -Some of the disadvantages of Python include its slow speed and heavy memory usage. It also lacks support for mobile environments, database access, and multi-threading. However, it is a good choice for rapid prototyping, and is widely used in data science, machine learning, and server-side web development. For more information about Python, see the [official Python site](https://www.python.org/). \ No newline at end of file diff --git a/docs/guides/development/python/python-get-current-directory/index.md b/docs/guides/development/python/python-get-current-directory/index.md deleted file mode 100644 index f08d7c7604a..00000000000 --- a/docs/guides/development/python/python-get-current-directory/index.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -slug: python-get-current-directory -title: "How to Get the Current Directory in Python" -title_meta: "A Guide to Paths in Python: Get Working Directory" -description: 'This guide provides information on how to retrieve the current working directory in Python using the os module. It covers the os.getcwd() function and provides sample code to demonstrate its usage.' -authors: ["Martin Heller"] -contributors: ["Martin Heller"] -published: 2023-06-12 -keywords: ['python change working directory', 'python get current directory', 'python change directory', 'change directory python'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -Programs often need to look for, read, and write specific files in a special location. By convention, and with support from the operating system, programs maintain a current working directory used as the context for many of their files. There may also be fixed directories and files that are special to the program, such as the configuration files for editors and IDEs. - -## What is the Current Working Directory? - -In most operating systems, a directory is a container for files, and possibly for other directories, called subdirectories. In Unix-like systems, directories are treated as a special kind of file. - -Files contain data and/or code. In Unix-like systems, files can have special bits in their permissions that mark them as executable. In Windows systems, the file type determines whether a file is executable. - -Folders might seem like they are synonymous with directories, but they are usually considered the graphical representation of directories rather than the directories themselves. Paths are strings that describe the locations of directories and files in the context of the operating system’s file system. Paths can be absolute, or relative. - -The current working directory (CWD) associates a specific directory with a program and makes that directory the default location for all file operations. When a program reads or writes a file without specifying a directory in the path, it looks for the file in the current working directory. If the program specifies a relative path, such as `../my_file`, the path is calculated relative to the current working directory, in this case specifying the directory above the CWD. If the program specifies an absolute path, such as `/usr/bin/python3`, the CWD does not enter into the calculation of the location. - -The current working directory is independent of the executable file’s location. - -## How to Get and Change the Current Directory in Python - -Python programs share the operating system’s idea of the CWD with all other programs, even though Python as a whole is a system-agnostic language. Python has a module, `os`, that provides a portable way of using operating system-dependent functionality, and within that module, each system’s version of Python has an appropriate implementation for that system. - -### How to Get the Current Working Directory - -To get the current working directory in Python, import the `os` module and call the `os.getcwd()` method. Here’s an example: - -```output -Python 3.9.7 (default, Sep 3 2021, 09:29:02) -[GCC 9.3.0] on linux -Type "help", "copyright", "credits" or "license" for more information. ->>> import os ->>> cwd = os.getcwd() ->>> print(cwd) -``` - -In the example above, the `os.getcwd()` method returns a string representing the current working directory. - -### How to Change the Current Working Directory - -To change the current working directory in Python, import the `os` module and call the `os.chdir()` method. Here’s an example: - -```output -Python 3.9.7 (default, Sep 3 2021, 09:29:02) -[GCC 9.3.0] on linux -Type "help", "copyright", "credits" or "license" for more information. ->>> import os ->>> current_dir = os.getcwd() ->>> print("Current working directory:", current_dir) -Current working directory: /home/user ->>> os.chdir("my_directory") ->>> new_dir = os.getcwd() ->>> print("New current working directory:", new_dir) -New current working directory: /home/user/my_directory -``` - -#### Change the current working directory to the path - -The `os.fchdir()` is a method that can support specifying a file descriptor in Python. This method changes the current working directory to the directory associated with a given file descriptor. The file descriptor must refer to an opened directory, not an open file. - -The `os.fchdir()` method can raise `OSError` and its subclasses such as `FileNotFoundError`, `PermissionError`, and `NotADirectoryError`. Some possible reasons for `OSError` being raised include: - -- The specified file descriptor is not valid -- The specified file descriptor does not refer to a directory -- The process does not have permission to access the directory associated with the file descriptor -- The directory associated with the file descriptor has been removed while the file descriptor was still open - -In Python 3.6 and later versions, the `os.chdir()` method was updated to accept a path-like object as its argument in addition to a string representing the directory path. - -Following is an example that uses a path-like object to set the current working directory: - -```output -Python 3.9.7 (default, Sep 16 2021, 13:09:58) -[Clang 13.0.0 (clang-1300.0.29.3)] on darwin ->>> import os ->>> os.getcwd() -'/Users/myuser' ->>> os.chdir("..") ->>> os.getcwd() -'/Users' ->>> os.chdir("~/Pictures") -Traceback (most recent call last): - File "", line 1, in -FileNotFoundError: [Errno 2] No such file or directory: '~/Pictures' -``` - -From the above code, although the `~/Pictures` directory exists, changing to it works in the shell: - -```command -(base) username@hostname ~ % cd ~/Pictures -(base) username@hostname Pictures % -``` - -But, in Python, the expansion of `~` to the home directory path does not happen. When you try to change the current working directory to `~/Pictures`, Python interprets it as a literal string with `~` as a character and `Pictures` as a directory name and tries to change the working directory to a directory named `~` in the current directory, which does not exist. - -{{< note noTitle=true >}} -The directory path conventions are different on Windows, where you see back-slashes for directory separators and letter disk descriptors in the shell and file manager. For example, `c:\Users\User\Documents`. In Python, you still use forward slashes for directory separators. The forward slashes are converted to back slashes inside the os module to interact with the Windows operating system. -{{< /note >}} - -To resolve the `FileNotFoundError` above, use the `os.path.exists()` method as shown in the example below. This method finds out whether a path exists or not before attempting to open or access it. - -```output -Python 3.9.12 (main, Apr 5 2022, 01:53:17) -[Clang 12.0.0 ] :: Anaconda, Inc. on darwin -Type "help", "copyright", "credits" or "license" for more information. ->>> import os ->>> os.path.exists("~/Pictures") -False ->>> os.path.exists("./Pictures") -True ->>> os.getcwd() -'/Users/myuser' -``` - -From the above Python example, the `os.path.exists(path)` method returns `True` if the specified path exists and `False` for broken symbolic links. The method internally uses `os.stat()` to check for the existence of the file or directory, and `os.stat()` requires read and execute permissions on the file or directory. Hence, on some platforms, even if the file, or directory exists physically on the system, the method may return `False` due to permission issues. - - -## Conclusion - -As you see, you can get the current working directory in Python by importing the `os` module and calling the `os.getcwd()` method. You can change the working directory using the `os.chdir()` method, and you can test whether a path, either directory, or file, exists using the `os.path.exists()` method. diff --git a/docs/guides/development/python/python-ml/index.md b/docs/guides/development/python/python-ml/index.md deleted file mode 100644 index fe211d867f8..00000000000 --- a/docs/guides/development/python/python-ml/index.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -slug: python-ml -title: "Python in ML: Why You Should Use Python for Machine Learning" -title_meta: "Python Programming, Machine Learning, and AI" -description: 'This introduction to Python in ML and AI defines each one, examines their differences and discusses why Python is good for machine learning. ✓ Learn more!' -authors: ["John Mueller"] -contributors: ["John Mueller"] -published: 2021-12-03 -keywords: ['ai python code','python for ai','why is python used for machine learning'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -*Artificial Intelligence* (AI), the appearance or simulation of human intelligence within a machine, is popular today because it can solve a huge number of problems within specific categories using well-known algorithms. Machine Learning (ML) is a sub-category of AI that depends on large datasets to affect the computations of algorithms to allow a generalized result that an application can apply to unseen data. In other words, ML provides an experience-based method to use AI techniques to solve problems realistically. Python ML features handle the required amounts of data effectively, and offer access to a broad range of algorithms using libraries like scikit-learn and SciPy to solve the various ML problem categories. Python’s capabilities deliver in a manner that is understandable in AI Python code, and which reduces the learning curve that most developers face. - -## Python, AI, and Machine Learning - -It’s possible to divide AI into two general areas: applied, where the AI solves a specific problem like assembly line control or trading stocks, and generalized; where the AI gains experience through various means in the form of algorithm augmentation. Python ML is part of the generalized AI branch and requires the use of sufficient amounts of data to train the algorithm. This process uses example data to change the calculations of an algorithm to solve general problems. When using Python for AI, it’s possible to reduce training time using a GPU by adding library support for libraries like Keras and TensorFlow. After training, you can configure a Python ML solution to continue learning so the solution becomes increasingly better at performing a task over time. ML is currently used in: - -- Fraud detection -- Resource scheduling -- Complex analysis -- Automation -- Customer service -- Safety systems -- Machine efficiency -- Access control -- Predicting wait times - -There are more ML applications. The one thing that’s striking about most of them is that they’re mundane in nature. The applications that ML is best at solving are repetitive, not very exciting, and generally things that humans wouldn't want to do anyway. The ability of Python ML to speed calculations along is the reason why Python is used for machine learning applications. - -## Why is Python Good for ML? - -The easiest way to discover why Python is good for ML is to test it. Even if you can’t follow along in this section, you can see that using Python for ML is not difficult to implement by reading through the steps. - -1. [Install a copy of Python if necessary](https://docs.python.org/3/using/unix.html#getting-and-installing-the-latest-version-of-python). Check the installation by typing `python --version` and press **Enter**. The output shows the Python version number. - -1. [Install a copy of scikit-learn](https://scikit-learn.org/stable/install.html), if necessary. Check the installation by typing `pip show scikit-learn` and press **Enter**. The output shows the scikit-learn version. The [scikit-learn package](https://scikit-learn.org/stable/) is one of the essentials for machine learning because it provides data analysis capability. - -1. Open the Python interpreter by typing python and press **Enter** in a terminal window. The Python prompt appears. - -1. Type `from sklearn.datasets import load_iris` and press **Enter**. This English-like syntax imports a specific function, `load_iris()`, from the `sklearn.dataset` namespace. Python doesn't make it hard to find datasets for testing. The [scikit-learn package](https://scikit-learn.org/stable/datasets/toy_dataset.html) provides several datasets and if these aren’t enough, it’s possible to find [repositories for just about every other need](https://towardsdatascience.com/data-repositories-for-almost-every-type-of-data-science-project-7aa2f98128b). - -1. Type `data = load_iris()` and press **Enter**. The variable data now contains the iris dataset. No other language makes it nearly as straightforward to load data for testing and experimentation purposes. Using known datasets for testing and experimentation corrects errors in your code. - -1. Type `print(data.data.shape)` and press **Enter**. The size of the iris dataset, 150 rows by 4 columns, appears. Learning about the content of datasets and manipulating them in all the ways needed for machine learning, are fortes of Python. - -1. Type `quit()` and press **Enter**. The Python session ends. When you experiment with the iris dataset and discover more about it, it’s clear why you want Python for your machine learning needs. - -## Is Python Better Than Other Languages for ML? - -No single language is ultimately better than another overall. Languages have strengths in certain areas and Python has more than its share. R is Python’s major competitor in the ML arena. Compared to R, Python provides a complete language you can use for experimentation, modeling, and production systems. In addition, Python’s library offerings are more business and research-oriented than R’s. However, R holds a significant advantage in statistics, and some might say that its graphics libraries are better than Python’s for research purposes. Other popular languages used for machine learning are: - -- **Java/JavaScript**: Is used extensively in business environments, so business developers feel comfortable using it. - -- **Julia**: Provides high-performance numerical analysis and computation for large, complex computation-intensive problems. - -- **LISP**: Considered the most efficient and flexible ML language for solving specific problems, rather than creating a generalized result. - -Each language has advantages and disadvantages. Python outshines them in the general creation of ML applications. In addition, Python has a high appeal according to the [Tiobe index](https://www.tiobe.com/tiobe-index/), which provides a measure of overall language popularity. The use case scenario driving this popularity is ML, where [57 percent](https://linuxiac.com/python-the-most-popular-programming-language/) of data scientists and ML developers use it as their primary language. - -## Conclusion - -AI and ML are complex topics that require attention to detail to obtain a good result. Even if you choose to use another language for development, Python is the best choice for learning about both AI and ML because it’s not difficult to use and you can accomplish much, with little code. Fortunately, experimentation is available using Linode. Using the smallest Linode, you can create a Python setup in minutes and start experimenting the same day. Consequently, using Python saves time, money, and effort when you need to gain entrance to the most important new technologies today. diff --git a/docs/guides/development/python/python-sets/index.md b/docs/guides/development/python/python-sets/index.md deleted file mode 100644 index a31e09638a2..00000000000 --- a/docs/guides/development/python/python-sets/index.md +++ /dev/null @@ -1,552 +0,0 @@ ---- -slug: python-sets -title: "Getting Started with Python Sets and Python Set Operations" -title_meta: "Introduction to Python Sets and Python Set Operations" -description: "Learn what Python sets are and how to use them. Our comprehensive guide covers essential Python set operations like union, intersection, and difference." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2023-04-23 -keywords: ['python sets', 'python sets intersection', 'python sets operations'] -tags: ['python'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Python Documentation: Built-in Types — Set Types](https://docs.python.org/3/library/stdtypes.html#set-types-set-frozenset)' -- '[W3 Schools: Python Sets](https://www.w3schools.com/python/python_sets.asp)' -- '[Toward Data Science: 10 Things You Should Know About Sets in Python](https://towardsdatascience.com/10-things-you-should-know-about-sets-in-python-9902828c0e80)' -- '[Programniz: Python Sets](https://www.programiz.com/python-programming/set)' -- '[GeeksforGeeks: Sets in Python](https://www.geeksforgeeks.org/sets-in-python/)' ---- - -Pythons sets are unordered collections modeled on mathematical sets, in which elements are unique. Python sets support the logical operations of mathematical sets, like union, intersection, and difference. This example code declares a new set and adds the elements of a second set to it, showing that the updated set contains unique elements: - -```file {lang="python"} -example_set = {1, 3, 5, 7} - -example_set.update({1, 2, 3, 4}) - -print(example_set) -``` - -```output -{1, 2, 3, 4, 5, 7} -``` - -This guide explains: - -- [What Python sets are](#what-are-sets-in-python) and what kinds of elements they can contain - -- [How to fetch an element from a Python set](#fetch-from-a-set), check if an element exists in a set, and iterate through the elements of a set. - -- [How to create a Python set](#create-a-python-set) with set literals, the `set()` constructor function, and the set comprehension syntax. - -- [How to add elements to and remove elements from Python sets](#modify-a-set) - -- [How to combine sets](#combine-sets) with the union, intersection, difference, and symmetric difference Python set operations. - -- [How to compare Python sets](#checking-python-set-relationships) with the subset, superset, and disjoint relationships. - -## What Are Sets in Python? - -In Python, a set is an unordered, mutable collection of unique elements. Mathematical set operations like union, intersection, and difference can be performed on them. Two different sets can be compared with the subset, superset, and disjoint relations. - -This example set uses the *set literal* syntax and contains the even integers between 2 and 8: - - {2, 4, 6, 8} - -Sets can contain elements of different types. This set contains some integers and some strings: - - {2, "hello", 4, "world"} - -The members of a set must be *[hashable](https://docs.python.org/3/glossary.html#term-hashable)*. Hashable objects are generally *[immutable](https://docs.python.org/3/glossary.html#term-immutable)*. Hashable objects must implement the [`__eq__()` method](https://docs.python.org/3/reference/datamodel.html#object.__eq__), so two different hashable objects and can be compared to check if they are equal in value to each other. For example, two string objects can be compared to check if they are the same string. - -{{< note >}} -Technically, a hashable object's `__hash__()` function must always return the same value for the lifetime of the object, rather than the object itself being immutable. [This blog post](https://inventwithpython.com/blog/2019/02/01/hashable-objects-must-be-immutable/) describes why, in practice, hashable objects used with sets are also immutable. -{{< /note >}} - -Integers and strings are built-in Python types that are hashable and can be stored in a set. Mutable container types like lists, dictionaries, and sets are not hashable, because their contents can change. A [Python tuple](/cloud/guides/python-tuples) is hashable if each of its values is hashable. - -## Create a Python Set - -Python provides a few ways to create sets: - -- A Python set can be declared with the *set literal* syntax. A set literal consists of a comma-separated list of unique, hashable objects wrapped in curly braces. - - ```file {lang="python"} - example_set = {2, 4, 6, 8} - ``` - - An empty set cannot be expressed with the set literal syntax. An empty pair of curly braces `{}` represents an empty [Python dictionary](/cloud/guides/python-3-dictionaries). Use the `set()` constructor (below) to create an empty set. - -- The [`set()` constructor function](https://docs.python.org/3/library/stdtypes.html#set) accepts a Python list (or any other [iterable](https://docs.python.org/3/glossary.html#term-iterable) object) as input and returns a set with unique values from the list. The items in the input list/iterable must be hashable. - - ```file {lang="python"} - example_set = set([2, 4, 6, 8, 8, 6, 4, 2]) - print(example_set) - ``` - - ```output - {8, 2, 4, 6} - ``` - - If no argument is passed to the constructor function, an empty set is created: - - ```file {lang="python"} - example_set = set() - print(example_set) - ``` - - ```output - set() - ``` - -- Like the `set()` constructor function, the [*set comprehension* syntax](https://docs.python.org/3.0/tutorial/datastructures.html?highlight=comprehension#sets) can be used to construct a set from another iterable. A set comprehension allows you to apply an expression to the elements of the iterable before they are added to the new set. A condition can also be enforced on the members of the iterable before they are included in the new set. - - The general syntax for using a set comprehension is: - - ```file {lang="python"} - { for in if } - ``` - - This example creates a set of all the even numbers from a list of even and odd integers: - - ```file {lang="python"} - even_integers_set = { x for x in [1, 2, 3, 4, 5] if x % 2 == 0 } - print(even_integers_set) - ``` - - ```output - {2, 4} - ``` - - This example multiplies the even integers by 2 when adding them to the new set: - - ```file {lang="python"} - double_even_integers_set = { x*2 for x in [1, 2, 3, 4, 5,] if x % 2 == 0 } - print(double_even_integers_set) - ``` - - ```output - {8, 4} - ``` - -## Python Set Operations - -The operations for Python sets differ from those available for lists and arrays. Lists and arrays are ordered collections, while the elements in a set are unordered. As a result, the methods for accessing elements and adding elements are different between sets and lists/arrays. As well, the mathematical operations of Python sets are not available for lists and arrays. - -{{< note >}} -The Linode docs library has guides for learning about Python lists and arrays: - -- [Python Lists and How to Use Them](/cloud/guides/python-lists-and-how-to-use-them) -- [Python Arrays: What They Are and How to Use Them](/cloud/guides/python-arrays) -{{< /note >}} - -## Fetch from a Set - -Because Python sets are unordered, elements cannot be accessed based on an index into the set. Instead, elements can be accessed using the `pop` method. This method returns an arbitrary value from the set while. **Note that the returned item is removed from the set.** - -```file {lang="python"} -example_set = {"a", 2, "c", 4, "b", 6} - -popped_element = example_set.pop() -print(popped_element) -print(example_set) -``` - -```output -2 -{'b', 4, 6, 'c', 'a'} -``` - -You can check if a value exists in a set by using Python's `in` operator. Expressions with this operator return `True` if the provided value is found in the set: - -```file {lang="python"} -example_set = {"a", 2, "c", 4, "b", 6} -if 4 in example_set: - print("Match found!") -``` - -```output -Match found! -``` - -{{< note >}} -Python sets are implemented with a form of [hash table](https://en.wikipedia.org/wiki/Hash_table), so checking if a value exists within a set is an efficient operation. In comparison, to check if an value exists in a list or array, the list/array must be looped through or sorted. -{{< /note >}} - -Like lists and arrays, Python sets can be looped through. Use a `for` loop to iterate through the elements in a set: - -```file {lang="python"} -example_set = {"a", 2, "c", 4, "b", 6} -for item in example_set: - print(item) -``` - -```output -2 -c -b -4 -6 -a -``` - -As an alternative to using the `pop` method, a `for` loop with a `break` statement can be used to retrieve an arbitrary value from a set. By doing this, the value is not removed from the set (unlike with the `pop` method, which does remove the value): - -```file {lang="python"} -example_set = {"a", 2, "c", 4, "b", 6} -for value in example_set: - break - -print(value) -print(example_set) -``` - -```output -2 -{2, 'c', 'b', 4, 6, 'a'} -``` - -## Modify a Set - -Python sets are *mutable*, meaning items can be added to and removed from them as needed. - -### Add an Element to a Set - -To add an element to a Python set, use the `add` method. The `add` method accepts a single new element to add to the set: - -```file {lang="python"} -example_set = {"this", "is", "a"} -example_set.add("set") -print(example_set) -``` - -```output -{'a', 'this', 'is', 'set'} -``` - -If the new element is already in the set, then the `add` method has no effect: - -```file {lang="python"} -example_set = {"this", "is", "a", "set"} -print(example_set) -example_set.add("set") -print(example_set) -``` - -```output -{'a', 'this', 'is', 'set'} -{'a', 'this', 'is', 'set'} -``` - -### Add Multiple Elements to a Set - -To add multiple values to a set at the same time, use the `update` method. The `update` method takes one (or more) Python iterables (lists, arrays, sets) as arguments. The `update` method adds any values that are not already in the set. Only unique values are added to the set. - -This example adds elements from a list to a set: - -```file {lang="python"} -example_set = {1, 3, 5, 7} -example_set.update([1, 2, 3, 4]) - -print(example_set) -``` - -```output -{1, 2, 3, 4, 5, 7} -``` - -Multiple arguments can be passed to `update`. This example passes two arguments: a list and another set. The elements from the list and the other set are added to the original set: - -```file {lang="python"} -example_set = {1, 3, 5, 7} -example_set.update([1, 2, 3, 4], {'a', 'b', 'c'}) - -print(example_set) -``` - -```output -{1, 2, 3, 4, 5, 7, 'b', 'a', 'c'} -``` - -Individual new elements can also be passed as arguments. This example adds elements from a list and three individual elements to a set: - -```file {lang="python"} -example_set = {1, 3, 5, 7} -example_set.update([1, 2, 3, 4], 'a', 'b', 'c') - -print(example_set) -``` - -```output -{1, 2, 3, 4, 5, 7, 'b', 'a', 'c'} -``` - -### Remove an Element from a Set - -In the [Fetch from a Set](#fetch-from-a-set) section, the `pop` method was demonstrated, which removes an arbitrary element from a set. Python sets also have two methods for removing specific values: - -The `remove` method accepts an element to be removed from the set: - -```file {lang="python"} -example_set = {1, 3, 5, 7} -example_set.remove(3) -print(example_set) -``` - -```output -{1, 5, 7} -``` - -If the value that is passed is not in the set, then a `KeyError` is thrown: - -```file {lang="python"} -example_set = {1, 3, 5, 7} -example_set.remove(2) -``` - -```output -Traceback (most recent call last): - File "", line 1, in -KeyError: 2 -``` - -The `discard` method works like the `remove` method, but it does not throw an error when the passed value is missing from the set: - -```file {lang="python"} -example_set = {1, 3, 5, 7} -example_set.discard(1) -example_set.discard(2) # no KeyError is thrown - -print(example_set) -``` - -```output -{3, 5, 7} -``` - -## Combine Sets - -One of the benefits of Python sets is their ability to use mathematical set operations like union, intersection, and difference. The next sections cover each of the logical operations available for Python sets. These operations can be implemented using either a method call or an operator, both of which are shown. - -The examples that follow use two example sets: - -```file {lang="python"} -mammal_set = {"wolves", "giraffes", "anteaters", "armadillos"} -insectivore_set = {"frogs", "lizards", "anteaters", "armadillos"} -``` - -`mammal_set` is a set of four mammals and `insectivore_set` is a set of four insectivores (animals that eat insects). - -### Python Set Union - -The *union* of two different sets is a new set that includes all of the contents of the constituent sets. - -In the Venn diagram below there are two overlapping circles, corresponding to the `mammal_set` and `insectivore_set` sets. The shaded green area represents the union of the `mammal_set` and `insectivore_set` sets. - -![Venn diagram of set union](python-sets-venn-union.png "Venn diagram of union of mammal_set and insectivore_set sets. All elements are in the diagram are shaded green, indicating that they are all included in the union.") - -The Python set `union` method is called on one set and takes another set as an argument. It returns the new union of those two sets: - -```file {lang="python"} -mammals_and_insectivores_set = mammal_set.union(insectivore_set) -print(mammals_and_insectivores_set) -``` - -```output -{'lizards', 'anteaters', 'giraffes', 'wolves', 'frogs', 'armadillos'} -``` - -Calling the `union` method on the `insectivore_set` set instead is equivalent to the previous example: - -```file {lang="python"} -mammals_and_insectivores_set = insectivore_set.union(mammal_set) -print(mammals_and_insectivores_set) -``` - -```output -{'lizards', 'anteaters', 'giraffes', 'wolves', 'frogs', 'armadillos'} -``` - -The union *operator* ` | ` also returns the union of two different sets. This is equivalent to using the `union` method: - -```file {lang="python"} -mammals_and_insectivores_set = mammal_set | insectivore_set -print(mammals_and_insectivores_set) -``` - -```output -{'lizards', 'anteaters', 'giraffes', 'wolves', 'frogs', 'armadillos'} -``` - -### Python Set Intersection - -The *intersection* between two sets includes the elements that are in both sets. - -In the Venn diagram below there are two overlapping circles, corresponding to the `mammal_set` and `insectivore_set` sets. The shaded green area represents the intersection for the `mammal_set` and the `insectivore_set` set: - -![Venn diagram of set symmetric difference](python-sets-venn-intersection.png "Venn diagram of intersection between the mammal_set and insectivore_set sets. Elements that are in both mammal_set and the insectivore_set in the diagram are shaded green, indicating that they are all included in the intersection.") - -The Python set `intersection` method is called on one set and takes another set as an argument. It returns the intersection between the two sets. This code creates a set of mammals that are insectivores, as in the diagram above. It excludes mammals that do not eat insects and non-mammals that do: - -```file {lang="python"} -mammals_that_are_insectivores_set = mammal_set.intersection(insectivore_set) -print(mammals_that_are_insectivores_set) -``` - -```output -{'anteaters', 'armadillos'} -``` - -{{< note >}} -Changing which set is passed as an argument to `intersection` does not change the returned value. -{{< /note >}} - -The intersection *operator* ` & ` also returns the intersection between two sets. This is equivalent to using the `intersection` method: - -```file {lang="python"} -mammals_that_are_insectivores_set = mammal_set & insectivore_set -print(mammals_that_are_insectivores_set) -``` - -```output -{'anteaters', 'armadillos'} -``` - -### Python Set Difference - -The *difference* between two sets includes the elements from the first set that are *not* in the second set. The overlap between the two sets is excluded. - -In the Venn diagram below there are two overlapping circles, corresponding to the `mammal_set` and `insectivore_set` sets. The shaded green area represents the difference between the `mammal_set` and the `insectivore_set` set: - -![Venn diagram of set difference, 1](python-sets-venn-difference-1.png "Venn diagram of difference between the mammal_set and insectivore_set sets. Elements that are in mammal_set that are not in insectivore_set in the diagram are shaded green, indicating that they are all included in the difference.") - -There is also a second difference between the sets, where the second set's elements are included instead of the first: - -![Venn diagram of set difference, 2](python-sets-venn-difference-2.png "Venn diagram of the second difference between the insectivore_set and mammal_set sets. Elements that are in insectivore_set that are not in mammal_set in the diagram are shaded green, indicating that they are all included in the difference.") - -The Python set `difference` method is called on one set and takes another set as an argument. It returns the difference between the two sets, where the argument set is subtracted from the set that the method is called on. This code creates a set of mammals that are not insectivores, as in the diagram above: - -```file {lang="python"} -mammals_that_are_not_insectivores_set = mammal_set.difference(insectivore_set) -print(mammals_that_are_not_insectivores_set) -``` - -```output -{'wolves', 'giraffes'} -``` - -The order that the sets are listed in the code matters, and reversing the order produces a different result. The code below instead calls `difference` on the `insectivore_set` and passes `mammal_set` as the argument. The returned difference is the set of insectivores that are not mammals: - -```file {lang="python"} -insectivores_that_are_not_mammals_set = insectivore_set.difference(mammal_set) -print(insectivores_that_are_not_mammals_set) -``` - -```output -{'frogs', 'lizards'} -``` - -The difference *operator* ` - ` also returns the difference between two sets. This is equivalent to using the `difference` method: - -```file {lang="python"} -# Same as mammal_set.difference(insectivore_set) -insectivores_that_are_not_mammals_set = mammal_set - insectivore_set -print(insectivores_that_are_not_mammals_set) -``` - -```output -{'wolves', 'giraffes'} -``` - -### Python Set Symmetric Difference - -The *symmetric difference* between two sets includes the elements from either set that are *not* in both sets. In other words, the symmetric difference is the union of two sets, minus their intersection. It can also be thought of as the union of the two differences between the sets. - -In the Venn diagram below there are two overlapping circles, corresponding to the `mammal_set` and `insectivore_set` sets. The shaded green area represents the symmetric difference for the `mammal_set` and the `insectivore_set` set: - -![Venn diagram of set symmetric difference](python-sets-venn-symmetric-difference.png "Venn diagram of symmetric difference between the mammal_set and insectivore_set sets. Elements that are not in both mammal_set and the insectivore_set in the diagram are shaded green, indicating that they are all included in the symmetric difference.") - -The Python set `symmetric_difference` method is called on one set and takes another set as an argument. It returns the symmetric difference between the two sets. This code creates a set of mammals that are not insectivores and insectivores that are not mammals, as in the diagram above: - -```file {lang="python"} -symmetric_difference_mammal_insectivores = mammal_set.symmetric_difference(insectivore_set) -print(symmetric_difference_mammal_insectivores) -``` - -```output -{'giraffes', 'frogs', 'lizards', 'wolves'} -``` - -{{< note >}} -Changing which set is passed as an argument to `symmetric_difference` does not change the returned value. -{{< /note >}} - -The symmetric difference *operator* ` ^ ` also returns the symmetric difference between two sets. This is equivalent to using the `symmetric_difference` method: - -```file {lang="python"} -symmetric_difference_mammal_insectivores = mammal_set ^ insectivore_set -print(symmetric_difference_mammal_insectivores) -``` - -```output -{'giraffes', 'frogs', 'lizards', 'wolves'} -``` - -## Checking Python Set Relationships - -Another way that Python sets resemble mathematical sets is that they support the subset, superset, and disjoint comparative operations. Each of these operations allows you to check on the relationship between two sets. - -### Python Subset - -A set is a *subset* of another set is a set if all the elements in the first set are also in the second set. The `issubset` method can be used to verify if a Python set is a subset of another set. - -In this example, `issubset` is called on `set_A`, and `set_B` is passed as an argument. The method returns True because `set_A` is a subset of `set_B`: - -```file {lang="python"} -set_A = {"a", "b", "c"} -set_B = {"a", "b", "c", "d", "e", "f", "g"} - -set_A.issubset(set_B) -``` - -```output -True -``` - -### Python Superset - -A set is a *superset* of another set is a set if the first set contains all the elements in the second set. The `issuperset` method can be used to verify if a Python set is a superset of another set. - -In this example, `issuperset` is called on `set_B`, and `set_A` is passed as an argument. The method returns True because `set_B` is a superset of `set_A`: - -```file {lang="python"} -set_A = {1, 2, 3, 4} -set_B = {1, 2, 3, 4, 5, 6, 7, 8, 9, 0} - -set_B.issuperset(set_A) -``` - -```output -True -``` - -### Python Disjoint Sets - -Two sets are *disjoint* if they share no elements in common. The `isdisjoint` method can be used to verify if a Python set is disjoint with another set. - -In this example, `isdisjoint` is called on `set_A`, and `set_B` is passed as an argument. The method returns True because `set_A` does not contain any of the elements in `set_B`, and `set_B` does not contain any of the elements in `set_A`: - -```file {lang="python"} -set_A = {"a", "b", "c", "d"} -set_B = {1, 2, 3, 4} - -set_A.isdisjoint(set_B) -``` - -```output -True -``` - -## Conclusion - -By following and referencing this guide, you can now get started using sets in your applications. Have more questions or want some help getting started? Consider searching for or asking your questions on the [Linode Q&A forum](https://www.linode.com/community/questions/). \ No newline at end of file diff --git a/docs/guides/development/python/python-string-interpolation/index.md b/docs/guides/development/python/python-string-interpolation/index.md deleted file mode 100644 index 0945aa5b34b..00000000000 --- a/docs/guides/development/python/python-string-interpolation/index.md +++ /dev/null @@ -1,251 +0,0 @@ ---- -slug: python-string-interpolation -title: "Python String Interpolation" -title_meta: "An Introduction to Python String Interpolation" -description: 'Python 3 string interpolation provides string substitution and string formatting. This guide covers the str.format() method, the modulo operator, f-strings, and the Template class.' -authors: ["John Mueller"] -contributors: ["John Mueller"] -published: 2022-05-20 -keywords: ['python string interpolation','python3 string format','python string format example'] -tags: ['python'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -During *string interpolation* a string literal is evaluated and if any placeholders are present, they are substituted by the indicated variable values. String interpolation helps reduce repetition in code and allows for dynamic string substitution. Python string interpolation consists of two parts: a *string template* and a value, or set of values, that you want to place within that template. To trigger string interpolation, the compiler needs some method of determining where to place the values within the string template. Python provides four techniques to do this, and this guide covers them all. Each string interpolation technique has its own advantages and disadvantages. The four ways to trigger string interpolation are the following: - -- A modulo character (`%`) placed within a string literal indicates where to make the string replacement and can be followed by string format indicators and variable names that store replacement values. -- Call the `.format()` method on your string literal. Use a pair of curly braces (`{}`) to indicate where to make the string replacement. The `.format()` method's arguments indicate the values to use for the string replacement. -- Precede any string literal by the letter `f` to use formatted string literals, also referred to as "f-strings". The string literal can contain curly braces (`{}`) that delimit a python expression and variables to use for the string substitution. -- Import the [String module's Template class](https://docs.python.org/3/library/string.html#template-strings) into your Python code. Store your string template in a variable using the `Template()` class. Then, use the Template class' `.substitute()` function to indicate which values to use in the string substitution. - -The sections below cover each of the four ways you can use string interpolation in Python. - -## The String Modulo Operator and String Formatting - -If code readability is a top concern, the modulo operator (`%`) method for string interpolation and formatting is a good choice. Its syntax is more concise compared to other string interpolation methods, because it does not require as many formatting arguments. The modulo operator is, however, the least flexible of the methods available for string interpolation. It is also an older string formatting method that may not remain supported as newer versions of Python are released. If you are working with an older Python codebase that uses the string modulo operator, then you may need to become familiar with its syntax. - -The general syntax for string formatting with the string modulo operator is the following: - - %[flags][width][.precision]format_indicator %(values) - -### Using String Format Indicators - -String formatting with the modulo operator includes the `%` character and a format indicator for each of the entries in the string template. A format indicator converts a provided value into the type indicated by the format indicator. The conversion is done before the value is inserted into the string. Python provides the following format indicators: - -- `%s`: String (performed using the `str()` function) -- `%d`: Integer -- `%f`: Floating point -- `%e`: Lowercase exponent -- `%E`: Uppercase exponent -- `%x`: Lowercase hexadecimal -- `%X`: Uppercase hexadecimal -- `%o`: Octal -- `%r`: Raw (performed using the `repr()` function) -- `%g`: Floating point for smaller numbers, lowercase exponent for larger numbers -- `%G`: Floating point for smaller numbers, uppercase exponent for larger numbers -- `%a`: ASCII (performed using the `ascii()` function) -- `%c`: Converts an `int` or a `char` to a character, such as `65` to the letter `A` - -The values to use for string substitution are placed after another `%` outside of the string literal. They can appear alone when working with a single value, or within a tuple when working with multiple values. You can use arithmetic calculations to generate a string replacement value, as demonstrated in the example below. - - myVal1 = 10.1 - myVal2 = 2.2 - print("The sum of %s and %s is %s." %(my_val_1, my_val_2, my_val_1+my_val_2)) - print("The sum of %d and %d is %d." %(my_val_1, my_val_2, my_val_1+my_val_2)) - print("The sum of %f and %f is %f." %(my_val_1, my_val_2, my_val_1+my_val_2)) - print("The sum of %e and %e is %E." %(my_val_1, my_val_2, my_val_1+my_val_2)) - print("The sum of %x and %x is %X." %(int(my_val_1), int(my_val_2), - int(my_val_1+my_val_2))) - print("The sum of %o and %o is %o." %(int(my_val_1), int(my_val_2), - int(my_val_1+my_val_2))) - -The output shows the result of the various format indicators. - -{{< output >}} -The sum of 10.1 and 2.2 is 12.3. -The sum of 10 and 2 is 12. -The sum of 10.100000 and 2.200000 is 12.300000. -The sum of 1.010000e+01 and 2.200000e+00 is 1.230000E+01. -The sum of a and 2 is C. -The sum of 12 and 2 is 14. -{{}} - -The format indicators in the first `print()` statement convert `my_val_1`, `my_val_2`, and `my_val_1+my_val_2` into strings. The string values are then substituted where the `%` character is placed within the string literal. The subsequent format indicator examples perform similar substitutions, but instead convert the values to decimals, floating point numbers, exponents, hexadecimal numbers, and octal numbers. Notice that you must convert floating-point values to integer values when working with the `%x`, `%X`, or `%o` format indicators. This is why the `int()` method is used in the last two `print()` statements. - -The following code shows the difference between string output and raw output for the same variable value: - - my_val = 'Hello There!' - print("%s, it's a lovely day!" %my_val) - print("%r, it's a lovely day!" %my_val) - -When working with raw output, the compiler doesn't interpret any of the characters and displays every character that is present in your string. This allows you to use special characters as needed. The second `print()` statement made use of the raw output format indicator (`%r`), so the output displays the opening and closing single quotes. - -{{< output >}} -Hello There!, it's a lovely day! -'Hello There!', it's a lovely day! -{{}} - -### Using Flags to Modify Output - -The string modulo operator supports flags that further control a string's formatting. The following examples demonstrate the usage of flags in modulo string formatting. - -The examples below use the variables, `my_val_1` and `my_val_2` to store the values to embed within the example strings. - - my_val_1 = 10.1 - my_val_2 = 2.2 - -The format indicator `%5d`, uses the `5` flag to indicate that the integers should be formatted with 5 spaces. Since the `d` portion converts the floating-point numbers to integers, three spaces are placed in front of the integers. - - print("%5d %5d" %(my_val_1, my_val_2)) - -{{< output >}} - 10 2 -{{}} - -Similarly, you can pad your output with zeros: - - print("%05d %02d" %(my_val_1, my_val_2)) - -{{< output >}} -00010 02 -{{}} - -To left justify your integers, use the `-` flag with your string formatting: - - print("%-5d %-5d" %(my_val_1, my_val_2)) - -{{< output >}} -10 2 -{{}} - -It’s possible to achieve more sophisticated formatting using the modulo operator. For example, you can format a floating-point value to have a specific number of digits after the decimal point. You can also add a dollar sign to monetary values. The following code shows both methods. - - print("%07.2f" %my_val_1) - print("$%1.2f" %my_val_1) - -{{< output >}} -0010.10 -$10.10 -{{}} - -In the first example, the first part of the format specifier tells the compiler to make the entirety of the output seven digits long (`%07`). The second part of the format specifier tells the compiler to keep the part after the decimal point limited to two digits (`2f`). The second example uses a value of one for the entirety of the output, but limits the decimal output to two digits. When you specify a length that can’t possibly hold the entire value, Python still displays the whole value as shown. - -## The Python String .format() Method and Positional Arguments - -Python's `str.format()` method provides another option for working with string formatting and interpolation. With the `str.format()` method, it’s possible to obtain any string output you need, however, it may require complex code. Due to its complexity, the `.format()` method may not always be the best choice. Formatting characters can prove difficult to read and even harder to troubleshoot. - -The `str.format()` method can be called on any Python string object. *Replacement fields* are surrounded by curly braces (`{}`). A replacement field contains the name of a [keyword argument](https://docs.python.org/3/glossary.html#term-argument) or the numeric index of a positional argument. - -The example below demonstrates using the `str.format()` with replacement fields and a named keyword argument. - - 'My name is {name} and I am {age} years old'.format(name= 'Frida', age=114) - -The Python interpreter returns the original string. However, the replacement fields are expanded to display the values of the `.format()` method's keyword arguments. - -{{< output >}} -'My name is Frida and I am 114 years old' -{{}} - -The next example demonstrates using the `str.format()` with replacement fields and the numeric index of a positional argument. - - 'My name is {0} and I am {1} years old'.format('Frida', 114) - -The Python interpreter returns the original string. In this case, the replacement fields are expanded to display the values of the `str.format()` method's positional arguments. - -Similar to the modulo method of string formatting, you can use format indicators with your `str.format()` method's arguments to further modify your string output. For example, the two variables, `my_val_1` and `my_val_2` store floating-point numbers. - - my_val_1 = 10.1111 - my_val_2 = 2.2222 - -Use the `str.format()` method and format indicators to adjust the number of digits that are displayed after the decimal point. - - 'The first value equals {:.2f} and the second value equals {:.3f}.'.format(my_val_1, my_val_2) - -Python returns the following formatted string: - -{{< output >}} -'The first value equals 10.11 and the second value equals 2.222.' -{{}} - -The format indicator `{:.2f}` specifies that the floating-point value should display two places after the decimal point. While the format indicator `{:.3f}`, specifies that the floating-point value should display three places after the decimal point. - -The `str.format()` method supports many format indicators that align your output, and adjust the [presentation type for string, integer, float, and decimal values](https://docs.python.org/3/library/string.html#format-specification-mini-language). - -## Python f-strings - -Another way to format strings is with Python f-strings, also known as *formatted string literals*. As with the other string formatting techniques discussed in this guide, f-strings can contain replacement fields denoted with curly braces (`{}`). When compared to the `str.format()` method, the f-string method produces simpler and more readable code. In addition, f-strings are more performant than the modulo operator or the `str.format()` method. - -When using a Python f-string, you must prefix your string literal with `f` or `F`. For example: - - import datetime - f'The date today is {datetime.datetime.now():%B %d, %Y}' - -The Python interpreter returns the following string: - -{{< output >}} -'The date today is March 01, 2022' -{{}} - -The replacement field includes a Python expression and a date format specifier. To achieve the same result using the `str.format()` method, you need the following code: - - import datetime - 'The date today is {:%B %d, %Y}'.format(datetime.datetime.now()) - -The f-string version of the code is more succinct and readable than the `str.format()` version. Similar to `str.format()`, you should refer to Python's [Format Specification Mini-Language documentation](https://docs.python.org/3/library/string.html#formatspec) to learn all the ways that you can format strings using f-strings. - -## Python Template Strings - -The Python 3 [Template Class](https://docs.python.org/3/library/string.html#template-strings) is part of the String module. Compared to the other methods described in this guide, template strings provide simpler string substitution. However, it does not support string formatting, like f-strings, `str.format()`, and the string modulo operator. While this makes Template class strings less powerful, they are considered more secure. For this reason, the Template class is a good choice if you are working with user-generated strings. - -String substitutions are indicated using a `$` interpolation character. The `$` should be followed by the name of a dictionary key that has been passed as an argument to the Template class's `substitute()` method. The `substitute()` method requires a dictionary-like object with keys as its argument. The `Template()` class accepts the template string as its argument. The example below imports the Template class, stores a new instance of the Template class and the string template in a variable. The new Template class' `substitution()` method is called and it contains the substitution string to use. - - from string import Template - greeting = Template('Welcome, $name') - greeting.substitute(name='Frida!') - -The Python interpreter returns the interpolated string: - -{{< output >}} -'Welcome, Frida!' -{{}} - -The Python 3 Template class provides more readable code, especially when using a single template with various values stored in a [dictionary](/cloud/guides/python-3-dictionaries). For example: - -{{< file "~/home/username/template_example.py">}} -from string import Template - -names = [] -names.append(dict(first='Anais', last='Nin')) -names.append(dict(first='Octavia', last='Butler')) -names.append(dict(first='Frida', last='Kahlo')) - -greeting = Template('Welcome, $first $last') - -for name in names: - print(greeting.substitute(name)) -{{}} - -When you run the above Python file, the following interpolated strings are returned as output: - -{{< output >}} -Welcome, Anais Nin -Welcome, Octavia Butler -Welcome, Frida Kahlo -{{}} - -The code in the `template_example.py` file is straightforward and readable. You can see that the template to be used derives its substitution values from the dictionaries appended to the `names` list. Then, a concise for loop, calls the template, and uses the `substitute()` method to perform the template substitutions. - -## Conclusion - -Python provides multiple ways to format strings, each with its own advantages and disadvantages. The method you choose depends on your particular use case and your familiarity with each method. Overall, the f-string method provides a good combination of formatting power and readable code. However, you may consider the `str.format()` method or the Template string class if maintaining security with user-generated strings is a concern. Finally, the string modulo operator is a legacy method for string substitution and formatting. But, it is helpful to familiarize yourself with this method if you are working with a legacy Python codebase. - - - - - - - - - - - diff --git a/docs/guides/development/react/how-to-deploy-a-react-app-on-centos-8/index.md b/docs/guides/development/react/how-to-deploy-a-react-app-on-centos-8/index.md deleted file mode 100644 index 6c7d2493eab..00000000000 --- a/docs/guides/development/react/how-to-deploy-a-react-app-on-centos-8/index.md +++ /dev/null @@ -1,197 +0,0 @@ ---- -slug: how-to-deploy-a-react-app-on-centos-8 -title: "Deploying a React Application on CentOS 8" -title_meta: "How to Deploy a React Application on CentOS 8" -description: 'This guide will show you how to deploy a React application you may have written to a CentOS 8 Linode that is running a web serve such as Apache or NGINX.' -og_description: 'This guide will show you how to deploy a React application you may have written to a CentOS 8 Linode that is running a web serve such as Apache or NGINX' -authors: ["Linode"] -contributors: ["Linode"] -published: 2018-04-24 -modified: 2020-04-24 -keywords: ['react','reactjs','deploy','rsync'] -tags: ["web applications","apache","nginx","centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Deploying_a_React_Application_on_Centos_8_1200x631.png -external_resources: -- '[React - A JavaScript library for building user interfaces](https://reactjs.org/)' -- '[Deploy a React App with Sass Using NGINX](https://web.archive.org/web/20191130010415/http://zabana.me/notes/build-deploy-react-app-with-nginx.html)' -audiences: ["beginner"] -concentrations: ["Web Applications"] -languages: ["javascript"] -relations: - platform: - key: deploy-react-app - keywords: - - distribution: CentOS 8 -aliases: [] ---- - -## What is React? - -[React](https://reactjs.org/) is a popular JavaScript library for building user interfaces. While React is often used as a frontend for more complex applications, it's also powerful enough to be used for full client-side applications on its own. - -Since a basic React app is static (it consists of compiled HTML, CSS, and JavaScript files), it is easy to deploy from a local computer to a Linode using [Rsync](/cloud/guides/introduction-to-rsync). This guide shows how to set up your CentOS 8 Linode and local machine so that you can easily deploy your app whenever changes are made. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. Install and configure a [web server](/cloud/guides/web-servers) to host a website on your Linode. This guide's examples will use the Apache and NGINX web servers. Complete the steps in the [Installing Apache Web Server on CentOS 8](/cloud/guides/how-to-install-apache-web-server-centos-8) guide or the [Installing NGINX on CentOS 8](/cloud/guides/how-to-install-nginx-centos-8) guide. - -1. This guide assumes you already have a React app you'd like to deploy. If you don't have one, you can quickly bootstrap a project following the steps in the [Create an Example React App](#create-an-example-react-app) section of this guide. This step should be completed on your local system. - -1. Install the Rsync program on your Linode server. - - sudo yum install rsync - -1. Install [Git](/cloud/guides/how-to-configure-git) on your local computer if it is not already installed. - - sudo yum install git - -1. Install the SELinux core policy Python utilities if you have not already done so. This will give you the ability to manage SELinux settings in a fine-grained way. - - sudo yum install -y policycoreutils-python-utils - -## Configure your Linode for Deployment - -The steps in this section should be performed on your Linode. - -### Create your Host Directory - -1. If it does not yet exist, create your site's web root directory. Most of the time, it will be located in the `/var/www` directory. - - sudo mkdir -p /var/www/example.com - -1. Set permissions for the new directory to allow your regular user account to write to it: - - sudo chmod 755 -R /var/www/example.com - -1. The Rsync program will execute its commands as the user you designate in your deployment script. This user must be the owner of your site's web root. Replace `example_user` with your own user's name and `/var/www/example.com` with the location of your site's web root. - - sudo chown -R example_user:example_user /var/www/example.com - -1. Use SELinux’s `chcon` command to change the file security context for web content if you have not already done so. - - sudo chcon -t httpd_sys_content_t /var/www/example.com -R - sudo chcon -t httpd_sys_rw_content_t /var/www/example.com -R - -### Configure your Web Server - -In this section, you will update your web server configuration to ensure that it is configured to point to your site's web root. - -1. Update your configuration file to point to your site's web root. - - > **Apache** - - > Modify the `DocumentRoot` in your virtual host file with the path to your site's web root. - - > {{< file "/etc/apache2/sites-available/example.com.conf" aconf >}} - - ServerAdmin webmaster@example.com - ServerName example.com - ServerAlias www.example.com - DocumentRoot /var/www/example.com/ ## Modify this line as well as others referencing the path to your app - ErrorLog /var/www/example.com/logs/error.log - CustomLog /var/www/example.com/logs/access.log combined - - {{< /file >}} - - > **NGINX** - - > Modify the `root` parameter with the path to your site's web root. - - > {{< file "/etc/nginx/sites-available.example.com" nginx >}} - server { - listen 80; - listen [::]:80; - - root /var/www/example.com; ## Modify this line - index index.html index.htm; - - } - {{< /file >}} - -1. Open the firewall for traffic if you have not already done so. - - sudo firewall-cmd --zone=public --permanent --add-service=http - sudo firewall-cmd --zone=public --permanent --add-service=https - sudo firewall-cmd --reload - -1. Restart the web server to apply the changes. - - > **Apache** - - > sudo systemctl restart apache2 - - > **NGINX** - - > sudo systemctl restart nginx - -## Configure your Local Computer - -### Install the Node Version Manager and Node.js - -You will need Node.js installed on your local computer in order to build your React app prior to copying your site files to the remote Linode server. - -{{% content "how-to-install-nvm" %}} - -### Create an Example React App - -If you already have a React App that you would like to deploy to your Linode, you can skip this section. Otherwise, follow the steps in this section to create a basic React app using the [create-react-app](https://github.com/facebook/create-react-app#create-react-app--) tool. - -1. Use the Node Package Manager to create your React app. - - npm init react-app ~/my-app - -### Create your Deployment Script - -1. Navigate to your app's directory. Replace `~/my-app` with the location of your React app's directory. - - cd ~/my-app - -1. Using a text editor, create a deployment script called `deploy.sh` in your app's root directory. Replace the following values in the example file: - - `example_user` with the username of your limited user account. - - `example.com` with your Linode's fully qualified domain name (FQDN) or public IP address. - - `/var/www/example.com/` with the location of your site's web root. This is where all of your React app's local `build/` files will be copied to on the remote server. - - {{< file "~/my-app/deploy.sh" bash >}} -#!/bin/sh - -echo "Switching to branch master" -git checkout master - -echo "Building app" -npm run build - -echo "Deploying files to server" -rsync -avP build/ example_user@example.com:/var/www/example.com/ -echo "Deployment complete" -{{< /file >}} - - This script will check out the `master` branch of your project on Git, build the app using `npm run build`, and then sync the build files to the remote Linode using Rsync. If your React app was not built with `create-react-app`, the build command may be different and the built files may be stored in a different directory (such as `dist`). Modify the script accordingly. - - {{< note respectIndent=false >}} -If your React app's directory is not initialized as a Git repository, the command `git checkout master` will return a `fatal: not a git repository (or any of the parent directories): .git` error. However, the script will continue on to the next commands and the files should still be transferred to your remote Linode server. See our [Getting Started with Git](/cloud/guides/how-to-configure-git#use-git-with-a-local-repository) guide to learn how to initialize a Git repository. - {{< /note >}} - -1. Make the script executable: - - sudo chmod u+x deploy.sh - -1. Run the deployment script. Enter your Linode user's password when prompted by the script. - - ./deploy.sh - -1. In a browser, navigate to your Linode's domain name or public IP address. If the deploy was successful, you should see your React app displayed. - - ![View your example React app in a browser.](example-react-app.png) - -1. Make a few changes to your app's `src` directory and then re-run the `deploy` script. Your changes should be visible in the browser after reloading the page. - -## Next Steps - -Deployment can be a complex topic and there are many factors to consider when working with production systems. This guide is meant to be a simple example for personal projects, and isn't necessarily suitable on its own for a large scale production application. - -More advanced build and continuous integration tools such as [Jenkins](https://jenkins.io) or [Travis](https://travis-ci.org/) can be used to automate a more complicated deployment workflow. This can include running unit tests before proceeding with the deployment and deploying to multiple servers (such as test and production boxes). See our guide on [Jenkins](/cloud/guides/automate-builds-with-jenkins-on-ubuntu) to get started. diff --git a/docs/guides/development/version-control/how-to-remove-untracked-files-in-git/index.md b/docs/guides/development/version-control/how-to-remove-untracked-files-in-git/index.md deleted file mode 100644 index 2cf4e7dfafa..00000000000 --- a/docs/guides/development/version-control/how-to-remove-untracked-files-in-git/index.md +++ /dev/null @@ -1,433 +0,0 @@ ---- -slug: how-to-remove-untracked-files-in-git -title: "Remove Untracked Files in Git" -description: 'Learn how to remove untracked files in git using the git clean command.' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2022-03-18 -keywords: ['Git remove untracked files','Remove untracked files git','Git remove all untracked files','How to remove untracked files in git'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Git clean documentation](https://git-scm.com/docs/git-clean)' -- '[Git ignore documentation](https://git-scm.com/docs/gitignore)' ---- - -During the development process, programmers and other Git users often wind up with many old and unwanted files. These might include prototypes, test data, and computer-generated files. Although these files do not necessarily cause problems, deleting them increases efficiency and improves organization. The `git clean` command is the fastest, safest, and easiest way to delete these files. This guide explains how to use Git to remove untracked files and provides many examples demonstrating how to use `git clean`. - -## Introduction to Untracked Files and Git - -Before using the `git clean` command, it is necessary to understand what an untracked file is and why untracked files matter. In every Git project, there are several types of files. One major distinction is between tracked and untracked files. - -*Tracked files* have already been added to Git through the use of the `git add` command. After a file has been added to Git, Git is fully aware of it. It has knowledge of the file details and contents, and is able to restore this information on demand. Tracked files can be either modified or unmodified. A modified file can be staged, also using the `git add` command. So a tracked file might be in one of several states, but Git continues to monitor it. - -The other category of files are the *untracked files*, which have not yet been added to the repository. Git can determine that these files exist, but it does not know anything else about them. It does not track their contents in its internal database. Because Git is not actively monitoring these files, it cannot take any action on them. For example, it cannot restore or recover the contents of these files. These files remain untracked until they are added to Git using the `git add` command. - -One danger of having untracked files is that the information is not saved. In the event of a hard drive failure, or accidental deletion or overwrite, any data is permanently lost. Because Git never stored the file, it cannot recover it. - -Sometimes, of course, files remain untracked because they were never intended to be added to the system. Some examples might include experimental changes, test data, discarded prototypes, build artifacts, and deprecated files. Some applications also dump a large number of auto-generated files into the working directory. It would be ill-advised to add them to Git. - -These files can certainly be left in an untracked state, and they do not usually cause any problems. However, there is some risk to keeping old and unnecessary untracked files around, due to the following reasons: - -- It is possible to accidentally `add` and `commit` them, especially when using the wildcard `*` symbol. -- They clutter up a workspace and take up unnecessary hard drive space. This can lead to confusion when returning to a workspace later on. It might be difficult to remember what the files were used for and whether they are still important. -- They appear in the output of commands such as `git status` as untracked files. This makes it more difficult to tell what files are truly important. - -There are several alternatives to getting rid of these files. For instance, it is possible to simply delete them using the `rm` command. However, this can be time-consuming, and it is easy to accidentally delete the wrong file. Other alternatives like `git reset` have a wider scope and might also unintentionally roll back changes to tracked or committed files. - -Additionally, some files should not be tracked in Git, but they are still important, and should not be deleted. A good example is a `.cfg` file or build object files. In this case, the `.gitignore` file is used to tell Git to disregard this file. Any files matching an exclusion rule in `.gitignore` are not displayed in the output of `git status` and the various Git commands do not affect them. - -The `git clean` command is the easiest and most efficient method to remove untracked files in Git. This command is highly targeted, easy to use, and does not have undesirable side effects. It leverages the contents of the `.gitignore` file and does not delete ignored files unless specifically told to do so. However, it is still important to use this command with a high degree of caution. When a file is deleted, it is gone for good. Git cannot recover the contents. - -To summarize, all files in a Git repository should eventually be handled in one of the following three ways: - -- They can be added to the repository using `git add`. -- They can be added to the `.gitignore` file, causing Git to ignore them. -- They can be removed using some variant of the `git clean` command. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. It's also helpful to consult our guides on [Getting Started with Git](/cloud/guides/how-to-configure-git) and [How to Navigate the Linux Terminal and File System](/cloud/guides/linux-navigation-commands). - -1. **Optional** Git must already be installed on the Linode before trying out the examples in this guide. The `git` package is often already pre-installed. To see if it is present, run the command `git --version`. If Git is already installed, this command displays the current version. If Git has not already been installed, use the command `sudo apt install git` to install it. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you are not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## How to Remove Untracked Files Using Git Clean - -This section demonstrates how to remove untracked files in Git using the `git clean` command. This command has several options allowing users to control the behavior or output of the command. `git clean` also has an interactive mode, making it easy to selectively delete a subset of the files. - -Try out the examples in this section using a local Git repository. If the Linode does not already have a repository, you can create one using the `git init` command. Run the `init` command inside the base directory of the repository. - -Without additional options, the command `git clean` does not do anything. One of the `-i`, `-n`, or `-f` options must normally be appended. - -```command -git clean -``` - -{{< output >}} -fatal: clean.requireForce defaults to true and neither -i, -n, nor -f given; refusing to clean -{{< /output >}} - -`git clean -f` is probably the most widely-used alternative. It forces Git to remove all untracked files with no further chance to refine or alter the operation. When it is finished, it displays a summary of the files it deleted. This command is functionally equivalent to manually deleting the files. -{{< note type="alert" >}} -Always use `git clean` with caution. This operation cannot be undone. -{{< /note >}} - -The following example demonstrates how `git clean -f` handles a combination of untracked, and tracked but uncommitted files. Before the operation, `testfile1.txt` has been added but not committed. The two files `testfile2.txt` and `testfile3.txt` have not been added, so they are both still untracked in Git. - -```command -git status -``` - -{{< output >}} -Changes to be committed: -new file: testfile1.txt - -Untracked files: -testfile2.txt -testfile3.txt -{{< /output >}} - -Run `git clean` using the `-f` option to delete the two untracked files. - -```command -git clean -f -``` - -{{< output >}} -Removing testfile2.txt -Removing testfile3.txt -{{< /output >}} - -Use either the `git status` or `ls` command to verify the untracked files have been deleted. The `status` command confirms the tracked but uncommitted file has been left untouched. However, the untracked files are no longer there. - -```command -git status -``` - -{{< output >}} -Changes to be committed: -new file: testfile1.txt -{{< /output >}} - -`git clean -n` reduces the risks associated with the `clean` command. It lists all the files the command intends to delete, but does not actually delete them. It acts like a "test run" of the command and can be used to avoid accidental deletions. - -Starting with the same configuration, the `git clean -n` command lists the files Git "would remove" in an actual operation. - -```command -git clean -n -``` - -{{< output >}} -Would remove testfile2.txt -Would remove testfile3.txt -{{< /output >}} - -Afterwards, the files are still listed in `git status` as they have not really been deleted. - -```command -git status -``` - -{{< output >}} -... -Untracked files: -testfile2.txt -testfile3.txt -{{< /output >}} - -The `git clean -e` option allows users to enter a specific exclude pattern or filename. Files having this name or matching this pattern are not deleted. - -In the following example, there are two untracked files named `testfile4.md` and `testfile4.txt`. - -```command -git status -``` - -{{< output >}} -... -Untracked files: -testfile4.md -testfile4.txt -{{< /output >}} - -To ignore `testfile4.txt` during the deletion process, append the option `-e testfile4.txt` to the `clean` command. The `-f` option is still required to force the remaining deletions. - -```command -git clean -f -e testfile4.txt -``` - -{{< output >}} -Removing testfile4.md -{{< /output >}} - -As a result, the `md` file is deleted, but the `txt` file is not. - -{{< output >}} -... -Untracked files: -testfile4.txt -{{< /output >}} - -`git clean -d` is recursive and cleans both the current directory and any subdirectories. - -```command -git clean -f -d -``` - -{{< output >}} -Removing archive/testfile2.txt -Removing testfile4.txt -{{< /output >}} - -It is also possible to specify a directory to limit the scope of the `git clean` operation. The command only applies to untracked files in this directory. The following example applies `git clean` to any files in the `example` directory. - -```command -git clean -f -d example -``` - -`git config` allows users to change the default `git clean` behavior. This permits `git clean` to delete untracked files without appending the `-f` option. It effectively makes `git clean` equivalent to `git clean -f`. To add this option to the configuration file , use the command `git config clean.requireForce false`. See the Git documentation for more details about [Git Clean](https://git-scm.com/docs/git-clean). - -{{< note >}} -The `-q` option runs `git clean` in quiet mode. This means `git clean` does not report the files it removes, but it still displays any errors. -{{< /note >}} - -### How to Remove Untracked files Using Git Clean in Interactive Mode - -The `-i` option is used to run `git clean` in interactive mode. It allows users to more precisely select the files to delete. This is a good choice for situations where some, but not all, of the untracked files should be deleted. It is also good for those who want to be extra cautious when running this command. - -The `-i` option displays a menu listing all available options. It also lists the files currently scheduled for deletion. - -To enter inactive mode, run the `git clean -i` command. Git displays the main menu for interactive mode. - -```command -git clean -i -``` - -{{< output >}} -Would remove the following items: - testfile2.txt testfile3.md testfile3.txt -*** Commands *** - 1: clean 2: filter by pattern 3: select by numbers - 4: ask each 5: quit 6: help -What now> -{{< /output >}} - -The list of options are as follows. - -- **clean**: This deletes the untracked files in the list the same way `git clean -f` does. -- **filter by pattern**: This option allows users to enter exclusionary patterns. `git clean` ignores untracked files with names matching any of these patterns. For instance, the filter pattern `*.txt` tells `git clean` not to delete any untracked files ending with the `.txt` extension. -- **select by number**: This displays a numbered list of the untracked files. Users can use these numbers to select files for deletion. -- **ask each**: This iterates through the list of untracked files one-by-one and allows the user to choose whether or not to delete each file. -- **quit** -- **help** - -From the main menu, users can select one of the first four options to remove files, quit interactive mode, or display the help page. - -Option `1` acts exactly like `git clean -f`. It removes all untracked files and its use is self-explanatory. It is typically selected after narrowing the list of files down using one of the other methods. The character `c` also launches this option. - -The second option is `filter by pattern`. It allows users to enter a pattern. Any filenames matching this pattern are ignored. This constrains and potentially reduces the number of files the `clean` option removes. - -To use the `filter by pattern` option, starting from the main menu, follow these steps. - -1. Select option `2` or enter the `f` key. Git lists all the untracked files and asks for a further response. - - ```command - 2 - ``` - - {{< output >}} -testfile2.txt testfile3.md testfile3.txt -Input ignore patterns>> - {{< /output >}} - -2. Enter the pattern for Git to ignore. For example, to avoid deleting any Markdown files, enter the pattern `*.md`. The `*` character acts as a wild card. Any file matching this pattern is ignored while the user remains in interactive mode. Git removes the matching files from consideration and displays an updated list of eligible files. - - ```command - *.md - ``` - - {{< output >}} -testfile2.txt testfile3.txt -Input ignore patterns>> - {{< /output >}} - -3. At this point, add any other patterns for Git to ignore. When all patterns have been entered, use the **ENTER** key to return to the main menu. Git then lists all files scheduled for deletion and waits for further input. - - {{< output >}} -Would remove the following items: - testfile2.txt testfile3.txt -*** Commands *** - 1: clean 2: filter by pattern 3: select by numbers 4: ask each 5: quit 6: help -What now> - {{< /output >}} - -4. To delete the list of files, enter `1` or `c`. - - ```command - 1 - ``` - - {{< output >}} -Removing testfile2.txt -Removing testfile3.txt - {{< /output >}} - -It is often easier to specify the files to delete from a numbered list. Enter `3` or `s` to access the `select by numbers` menu. This option allows users to specify individual files, a range of files, or a wildcard `*` specifying all files. Only the intentionally selected files are eligible for deletion. - -Starting from the main interactive menu, follow these instructions to select and delete files by number. - -1. Enter option `3` or use the `s` key to enter the `select by numbers` menu. - - ```command - 3 - ``` - - {{< output >}} -1: testfile2.txt 2: testfile3.md 3: testfile3.txt -4: testfile4.txt -Select items to delete>> - {{< /output >}} - -2. Enter the items to delete, separating them using commas. A range can also be used to select multiple sequential items, using the format `start-end`. The following response selects items `1`, `2`, and `4`, but not item `3`. Git highlights the selected files using a `*`. - - ```command - 1-2,4 - ``` - - {{< output >}} -* 1: testfile2.txt * 2: testfile3.md 3: testfile3.txt -* 4: testfile4.txt -Select items to delete>> - {{< /output >}} - -3. Select **RETURN** to visit the main menu again. Git lists the selected files. - - {{< output >}} -Would remove the following items: - testfile2.txt testfile3.md testfile4.txt -*** Commands *** - 1: clean 2: filter by pattern 3: select by numbers - 4: ask each 5: quit 6: help -What now> - {{< /output >}} - -4. Enter `1` or `c` to remove the selected files. - - ```command - 1 - ``` - - {{< output >}} -Removing testfile2.txt -Removing testfile3.md -Removing testfile4.txt - {{< /output >}} - -The final method for deleting files is the `ask each` option. Enter either a `4` or an `a` to use this option. It lists all files, one at a time, and asks the user whether they should be deleted. Answer `y` to delete the file or `n` to skip the file. - -To use the `ask each` method, follow these steps. - -1. Use the `4` or `a` key to access the `ask each` option. - - ```command - 4 - ``` - -2. Git displays the first file and asks whether it should be deleted or not. - - {{< output >}} -Remove testfile2.txt [y/N]? - {{< /output >}} - -3. Enter `y` to add the file to the deletion list. - - ```command - y - ``` - -4. Enter `y` or `n` for each of the remaining files. - -5. After the user has reviewed every file, Git deletes all selected untracked files. - - {{< output >}} -Removing testfile2.txt - {{< /output >}} - -The remaining two options are self-explanatory. Option `5` (`q`) quits the interactive menu. Option `6` is the help menu, which can also be accessed using `h`. The help menu explains the various options. - -{{< output >}} -clean - start cleaning -filter by pattern - exclude items from deletion -select by numbers - select items to be deleted by numbers -ask each - confirm each deletion (like "rm -i") -quit - stop cleaning -help - this screen -? - help for prompt selection -{{< /output >}} - -## How to Remove Ignored Files Using Git Clean - -The `.gitignore` file specifies which untracked files should be ignored. This is the best option for handling configuration or system files that should not be checked-in nor deleted. Users can run the `ls` command and see these files, but they do not appear in the results of commands like `git status`. This file is often created automatically and is pre-loaded with a list of common extensions to ignore. See the documentation for more information on [Git Ignore](https://git-scm.com/docs/gitignore). - -`git clean` typically ignores the files and filename patterns listed in the `.gitignore` file. This matches the behavior of other Git commands, which also ignore these files by default. - -However, this behavior can be overridden using either the `-x` or `-X` option. These options tell Git not to follow the standard ignore rules. This means Git can consider any files covered by `.gitignore`. However, they differ in how they handle other untracked files. - -{{< note >}} -Git still respects any exclusion rules added with the `-e` option. -{{< /note >}} - -In the following example, any untracked `*.c` files for the project show up in `git status`. However, the `*.o` files do not. This is because the `.gitignore` file includes the pattern `*.o`. This tells Git to ignore files having this extension. - -```file {title=".gitignore" lang="aconf"} -*.o -``` - -Ordinarily, `git clean` does not affect untracked files that match `.gitignore` patterns. Use `git clean -n` to confirm `testfile3.o` is not under consideration for deletion. - -```command -git clean -n -``` - -{{< output >}} -Would remove testfile3.c -Would remove testfile3.txt -Would remove testfile4.txt -{{< /output >}} - -To remove only the ignored files, leaving other untracked files untouched, use `git clean -f -X`. The `-X` option removes the untracked `.o` files. Developers can use this option to force a perfectly clean build. - -```command -git clean -f -X -``` - -Git only removes the untracked files that match a pattern in `.gitignore`. - -{{< output >}} -Removing testfile3.o -{{< /output >}} - -To remove all untracked files, whether they are ignored or not, use the command `git clean -f -x` instead. The `-x` option removes all untracked files, including ignored files. - -```command -git clean -f -x -``` - -{{< output >}} -Removing testfile3.c -Removing testfile3.o -Removing testfile3.txt -{{< /output >}} - -## Concluding Thoughts about Removing Untracked Files in Git - -This guide discusses how to use `git clean` to remove untracked files in Git. Although old untracked files do not necessarily cause any harm, they clog up a work space and create a sense of clutter and chaos. There are several methods of removing them, but the easiest way to remove an untracked file in Git is the `git clean` command. This command deletes untracked files while leaving other files untouched. - -The `clean` command usually requires the force option `-f` to take effect. It also has a dry run mode and can operate recursively. `git clean` has a handy interactive mode that permits users to exclude files based on a pattern filter, or select them from a numbered list. By default, `git clean` does not delete any files matching the `.gitignore` file. However, one option targets only the untracked ignored files, while another acts on all untracked files. For more information on the `git clean` command, consult the [Git Documentation](https://git-scm.com/docs/git-clean). \ No newline at end of file diff --git a/docs/guides/development/version-control/how-to-switch-git-branches/index.md b/docs/guides/development/version-control/how-to-switch-git-branches/index.md deleted file mode 100644 index b46b64304fe..00000000000 --- a/docs/guides/development/version-control/how-to-switch-git-branches/index.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -slug: how-to-switch-git-branches -title: "Switching Git Branches" -title_meta: "How to Switch Git Branches" -description: 'A very basic guide on how to switch a branch in Git using Ubuntu 20.10, but for any operating system with the command line git tools installed.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2021-05-28 -keywords: ['git','branch','branches','switch branch','switch branches','switch git branches'] -tags: ["git", "version control"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Git.jpg -external_resources: -- '[Git](https://git-scm.com/)' -- '[Git Branching - Basic Branching and Merging](https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging)' ---- - -Git is a distributed version control system. It supports non-linear, distributed development, allowing multiple contributors to work on a project simultaneously within what are called *branches*. This guide walks you through the basics of what a branch is and how to switch between branches. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. This guide assumes you have Git installed, are familiar with the *command-line interface* (CLI) and Git, and have a project in Git. - -1. Update your local workstation's system using the tools and package managers for your operating system. - -## What is a Branch in Git? - -If you're doing work on a website, application, or other Git project, you probably don't want to push the code to production immediately. This is where creating a branch is necessary, as it marks what version of the code you branched from and allows you to work without breaking anything currently in production, or having you disturb other developers (or vice versa). - -## Why Would I Need to Switch Branches in Git? - -Different branches have different purposes, so sometimes you need to switch from one to another. For example, you may be working in two branches yourself, or you may be pair programming. Git was designed to anticipate the need to switch between branches with `git checkout` (`git switch` was introduced to be more intuitive and decrease the number of functions in `git checkout`, but it is currently considered [experimental](https://git-scm.com/docs/git-switch#_description) and not covered in this article). - -## Switching to an Existing Git Branch - -You may have multiple tasks you're working on in separate branches. Let's say one is a hotfix and one is a feature request. You realize you have something to add on the hotfix, so you need to switch to that branch in Git: - -1. Enter `git checkout quick-hotfix` at the command prompt. - -2. You should then get a response stating: - {{< output >}} -Switched to branch 'quick-hotfix' -{{< /output >}} - -At that point, you can complete the hotfix. - -## Switching to a New Git Branch - -Suppose you are updating your product's API and creating the branch you need to switch to. To do so: - -1. Enter a command similar to the following at the command prompt. The `-b` flag is followed by the name of the new branch. - - `git checkout -b update-api` - -2. You should then get a response stating: - {{< output >}} -Switched to a new branch 'update-api' -{{< /output >}} - -You can then commit work to that branch. - -## Switching to a Remote Git Branch - -Using remote branches is one of the best ways to work with your colleagues. Let's say you're pair programming for the day and your coworker has a branch called "edit-button-feature-request" ready for the two of you. To switch to a remote branch in Git: - -1. Enter `git checkout --track origin/edit-button-feature-request` at the command prompt. - -2. You should then get a response stating: - {{< output >}} -Branch edit-button-feature-request set up to track remote branch newsletter from origin. -Switched to a new branch 'edit-button-feature-request' -{{< /output >}} - -At that point, a new local branch is created, allowing you to start on that feature request, and push to that branch as necessary. - -## Further Reading on Git Branches - -Git branching is an extensive subject. For more on branching in Git, see ["Git Branching - Branches in a Nutshell" on git-scm.com](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell). diff --git a/docs/guides/development/version-control/how-to-unbundle-nginx-from-omnibus-gitlab-for-serving-multiple-websites/index.md b/docs/guides/development/version-control/how-to-unbundle-nginx-from-omnibus-gitlab-for-serving-multiple-websites/index.md deleted file mode 100644 index d1ce2e8f182..00000000000 --- a/docs/guides/development/version-control/how-to-unbundle-nginx-from-omnibus-gitlab-for-serving-multiple-websites/index.md +++ /dev/null @@ -1,190 +0,0 @@ ---- -slug: how-to-unbundle-nginx-from-omnibus-gitlab-for-serving-multiple-websites -title: 'Unbundling NGINX from Omnibus Gitlab' -title_meta: 'How to Unbundle NGINX from Omnibus GitLab' -description: 'Unbundle the default NGINX server from Omnibus Gitlab and install and configure your own to enable virtual hosting.' -authors: ["Jordi Bassagañas"] -contributors: ["Jordi Bassagañas"] -published: 2016-02-29 -modified: 2017-06-21 -keywords: ["version control", "git", "gitlab", "install gitlab on ubuntu", "how to manage repositories with gitlab"] -tags: ["version control system","email","ruby","nginx","ubuntu"] -license: '[CC BY-ND 4.0](http://creativecommons.org/licenses/by-nd/4.0/)' -aliases: [] -external_resources: - - '[Updating GitLab via Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/update.md)' - - '[Installing Passenger + NGINX](https://www.phusionpassenger.com/library/install/nginx/install/oss/trusty/)' -audiences: ["intermediate"] ---- - -Omnibus GitLab is a software package (or software stack) that allows you to easily install and run GitLab on your Linode. -This guide walks you through the process of installing and setting up your own NGINX server on a typical Omnibus installation. Using the method outlined here, you are not forced to use Omnibus's default settings, and can create as many virtual hosts as you need for hosting multiple websites and apps on the same server as your GitLab. - -Preconfigured software stacks sometimes bring a series of challenges to those who need to customize specific settings. If you require more control over your installation, consider [installing GitLab from source](/cloud/guides/install-gitlab-on-ubuntu-14-04-trusty-tahr). This application stack could benefit from large amounts of disk space, so also consider using our [Block Storage](https://techdocs.akamai.com/cloud-computing/docs/block-storage) service with this setup. - - -## Before You Begin - -1. Familiarize yourself with Linode's [Getting Started guide](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and complete the steps for setting your Linode's hostname and timezone. - -2. Complete the sections of our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to create a standard user account, harden SSH access and remove unnecessary network services. - -3. This guide has been tested with Ubuntu 14.04 LTS and 16.04 LTS. Some commands will be slightly different for each version, so be sure to read each step carefully for version-specific instructions. - -4. Update your system: - - sudo apt-get update && sudo apt-get upgrade -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, visit our [Users and Groups guide](/cloud/guides/linux-users-and-groups) for more information. -{{< /note >}} - -## Install Omnibus GitLab - -If you're already running an Omnibus GitLab environment [upgrade to the newest version](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/update.md) and proceed to the next section, [Unbundle NGINX from Omnibus](#unbundle-nginx-from-omnibus-gitlab). If you're installing GitLab for the first time, continue with the steps in this section. - -Note that NGINX cannot be disabled in older versions of GitLab Community Edition (CE). If you currently have an older version of GitLab CE installed, we recommend that you upgrade incrementally to avoid issues. - -1. Install the dependencies: - - sudo apt-get install curl openssh-server ca-certificates postfix - -2. While installing Postfix, you'll be asked to configure a few basic settings. On the first [ncurses](https://en.wikipedia.org/wiki/Ncurses) screen, select **Internet Site** as the mail configuration. On the second screen, enter your fully qualified domain name (FQDN). This will be used to send email to users when configuring new accounts and resetting passwords. The rest of the mail options will be configured automatically. - - {{% content "email-warning-shortguide" %}} - -3. Add the GitLab CE repository and install the `gitlab-ce` package: - - curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash - sudo apt-get install gitlab-ce - - You can view the contents of the script in its entirety on the [GitLab website](https://packages.gitlab.com/gitlab/gitlab-ce/install) if you're hesitant to run it sight-unseen. The [GitLab downloads page](https://about.gitlab.com/downloads/) also contains alternative download methods if you're still not comfortable running their script. - -## Unbundle NGINX from Omnibus GitLab - -1. To unbundle NGINX from GitLab, we'll need to disable the version included in the Omnibus package. Add the following lines to `/etc/gitlab/gitlab.rb`: - - {{< file "/etc/gitlab/gitlab.rb" >}} -# Unbundle NGINX from Omnibus GitLab -nginx['enable'] = false -# Set your NGINX's username -web_server['external_users'] = ['www-data'] - -{{< /file >}} - - -2. Reconfigure GitLab to apply the changes: - - sudo gitlab-ctl reconfigure - -For more information on how to customize Omnibus NGINX, visit the [official NGINX documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/nginx.md). - -## Install Ruby, Passenger, and NGINX - -Now that GitLab's bundled NGINX has been disabled, the next step is to install and configure the web server from scratch. - -1. Since GitLab is written in Ruby, install Ruby on your system: - - sudo apt-get install ruby - sudo gem install rubygems-update - sudo update_rubygems - -2. We'll also need to install [Phusion Passenger](https://en.wikipedia.org/wiki/Phusion_Passenger), a web application server for Ruby. Install Phusion Passenger's PGP key: - - sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7 - -3. Add Passenger's APT repository by adding the following lines to `/etc/apt/sources.list.d/passenger.list`: - - {{< file "/etc/apt/sources.list.d/passenger.list" >}} -deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main - -{{< /file >}} - - - {{< note respectIndent=false >}} -If you're using Ubuntu 16.04, replace `trusty` with `xenial` in the above command. -{{< /note >}} - -4. Update your package repositories: - - sudo apt-get update - -5. Install Passenger and NGINX: - - sudo apt-get install nginx-extras passenger - -6. Enable the new Passenger module by uncommenting the `include /etc/nginx/passenger.conf;` line from the `/etc/nginx/nginx.conf` file: - - {{< file "/etc/nginx/nginx.conf" aconf >}} -include /etc/nginx/passenger.conf; - -{{< /file >}} - - -4. Finally, restart NGINX. On Ubuntu 14.04: - - sudo service nginx restart - - On Ubuntu 16.04: - - sudo systemctl restart nginx - -For further information, please refer to [Installing Passenger + NGINX on Ubuntu 14.04 LTS (with APT)](https://www.phusionpassenger.com/library/install/nginx/install/oss/trusty/ "Installing Passenger + NGINX"). - -## Create a New Virtual Host - -In this section, we'll create a new virtual host to serve GitLab. Since we've unbundled NGINX, we'll also be able to configure other virtual hosts for other websites and apps. - -1. Copy the default virtual host file to a new virtual host file, replacing `example.com` with your virtual host: - - sudo cp /etc/nginx/sites-available/default /etc/nginx/sites-available/example.com - -2. Edit your new virtual host file to match the following, replacing `example.com` with your own hostname: - - {{< file "/etc/nginx/sites-available/example.com" >}} -upstream gitlab { - server unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket; -} - - server { - listen 80; - server_name example.com; - server_tokens off; # don't show the version number, a security best practice - root /opt/gitlab/embedded/service/gitlab-rails/public; - - # Increase this if you want to upload large attachments - # Or if you want to accept large git objects over http - client_max_body_size 250m; - - # individual nginx logs for this gitlab vhost - access_log /var/log/nginx/gitlab_access.log; - error_log /var/log/nginx/gitlab_error.log; - - location / { - proxy_redirect off; - proxy_set_header Host $http_host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Real-IP $remote_addr; - proxy_pass http://gitlab; - } -} - -{{< /file >}} - - -3. Enable your new virtual host by symbolically linking it to `sites-enabled` (change `example.com`): - - sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/example.com - -4. Restart NGINX to load your changes. On Ubuntu 14.04: - - sudo service nginx restart - - On Ubuntu 16.04: - - sudo systemctl restart nginx - -5. Since NGINX needs to access GitLab, add the `www-data` user to the `gitlab-www` group: - - sudo usermod -aG gitlab-www www-data - -Congratulations! You have turned a default Omnibus GitLab server into a multi-purpose one. To serve additional websites and apps using your newly unbundled NGINX server, simply create additional virtual hosts above, and configure them to your needs. For more information, please refer to our guide on [how to configure NGINX](/cloud/guides/how-to-configure-nginx). diff --git a/docs/guides/development/version-control/how-to-use-gitignore/index.md b/docs/guides/development/version-control/how-to-use-gitignore/index.md deleted file mode 100644 index 04ee3879980..00000000000 --- a/docs/guides/development/version-control/how-to-use-gitignore/index.md +++ /dev/null @@ -1,498 +0,0 @@ ---- -slug: how-to-use-gitignore -title: "Use .gitignore to Ignore Specific Files and Folders" -description: 'This guide explains the .gitignore file, describes how to create and update it, and documents its syntax' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2022-06-10 -keywords: ['gitignore','.gitignore','Gitignore syntax','How to add files to gitignore','What is gitignore'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[gitignore Documentation](https://git-scm.com/docs/gitignore)' -- '[Git website](https://git-scm.com/)' ---- - -[Git](https://git-scm.com/) is a powerful *version control system* (VCS). It allows developers to manage, coordinate, and control the contents of their workspaces, but is not without complexity. Git users often struggle with untracked local files that complicate the output of commands like `git status`. This guide explains the `.gitignore` file, which provides a handy workaround to this problem. It also describes how to create a `.gitignore` file, how to add files and folders to `gitignore`, and how to use its powerful syntax. - -{{< note >}} -Throughout this guide, `gitignore` refers to the `.gitignore` file. The full `.gitignore` name is always used in commands, outputs, and when referencing the full path of the file. -{{< /note >}} - -## What is gitignore? - -In a Git repository, most files are either tracked or untracked. But the `gitignore` file enables a third category of files. Here's an explanation of the three types of files in a Git repository: - -- **Tracked**: These files are already added/staged or committed to the repository. -- **Untracked**: These files are not yet staged or committed. The developer intends to stage or commit them at some later time. -- **Ignored**: These are untracked files that a developer does not want to stage or commit. Git has been told to ignore these files, so they do not appear in the input of Git commands. As far as Git is concerned, these files do not exist. - -To list the tracked and untracked files in a Git repository, use the `git status` command. It lists all tracked files that have changed, along with the untracked files. However, it does not list any ignored files or folders. These entities are hidden, and therefore no longer shown as untracked. This removes clutter from the Git commands and makes it easier to focus on changes to relevant files. - -To ignore a file or folder, add it to a file named `.gitignore`. This is a text file normally located in the root directory of a Git repository, although it can reside elsewhere. The preceding `.` character indicates `gitignore` is a hidden file. - -Git does not automatically create the `gitignore` file. It must be created manually. - -Each line in the file represents a different pattern, or rule, describing the files Git should ignore. The `gitignore` syntax includes a series of special operators for developing patterns with much larger scopes. Unfortunately, there is no Git command to create or edit the `gitignore` file. These actions must be performed manually. - -Developers should ignore files and folders they do not plan to push, rather than leaving them in an untracked state. This avoids confusion, reduces the chance of accidental commits, and helps developers structure their workplace. Here are some types of files that are good candidates for `gitignore`. - -- Object files and compiled code, such as `.o` files. -- Build output directories. -- Caches. -- System files. -- Auto-generated files, including `.lock` and `.tmp` files. -- Personal configuration or IDE files. -- Temporary test data for unit testing. -- Placeholder or stub files used during early development. -- Files containing sensitive information like passwords and keys. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. Ensure Git is installed on your Linode. For information on installing up Git, see the Linode guide to [installing Git](/cloud/guides/how-to-install-git-on-linux-mac-and-windows). Essentially: - - ```command {title="Debain / Ubuntu"} - sudo apt install git - ``` - - ```command {title="AlmaLinux / CentOS Stream / Fedora / Rocky Linux"} - sudo dnf install git - ``` - -1. To provide an environment for testing `gitignore` behavior, create and initialize a test Git repository: - - ```command - mkdir testgit - cd testgit - git init - ``` - -1. Create the example files and folders necessary to follow along with this guide: - - ```command - mkdir {subdir1,subdir2,subdir3} && touch 1.bak a.bin b.bin file1.txt file2.txt file3.txt file4.txt file5.txt file6.txt one.bak subdir1/file7.txt subdir2/file8.txt subdir3/files.log - ``` -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you are not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## How to Use the gitignore file - -`gitignore` is a normal text file. It contains a set of rules telling Git what files and folders to ignore. Users must create and edit it manually. The following sections explain how and where to create the file and how to ignore files and folders. The following examples use a sample Git repository named `testgit`. This repository has already been created using the `git init` command. - -This guide is optimized for Linux and Ubuntu users, but the Git commands are common to all platforms. The `gitignore` syntax is platform independent. - -### How to Create the gitignore File - -Most developers add the `gitignore` file to the root directory of the repository. However, it can be created in any directory. The patterns in a `gitignore` file are always relative to the location of the `gitignore` directory. It is also possible to create multiple `gitignore` files. The rules in each file are cumulative and are processed in a relative manner. - -There is no command for creating the `.gitignore` file. To create the `.gitignore` file, first, make sure you're in the root directory of the Git project (i.e `testgit`). Then use a text editor, or simply the `touch`command, to create the file: - -```command -touch .gitignore -``` - -### How to Add Files to gitignore - -The simplest use of `gitignore` is to ignore an individual file. Add the full name of the file to be ignored to the `.gitignore` file. Each new entry must appear on a separate line. - -Git ignores all files with this name no matter where they are located in the repository. A later section discusses how to ignore multiple files matching a pattern. Here are the steps required to add a file to `gitignore`. - -1. Run the `git status` command to review the list of untracked files. Determine which files are not important and should not be listed. - - ```command - git status - ``` - - {{< output >}} -Untracked files: - .gitignore - 1.bak - a.bin - b.bin - file1.txt - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - one.bak - subdir1/ - subdir2/ - subdir3/ -{{< /output >}} - -1. Edit the `.gitignore` file: - - ```command - nano .gitignore - ``` - -1. To ignore `file1.txt`, add the full name of the file to `gitignore`: - - ```file {title="testgit/.gitignore"} - file1.txt - ``` - - {{< note respectIndent=false >}} -This pattern ignores any file named `file1.txt` anywhere in the Git repository. -{{< /note >}} - -1. Press **CTRL+X** to exit nano, **Y** to save, and **Enter** to confirm. - -1. Run `git status` again and confirm `file1.txt` is no longer listed amongst the untracked files: - - {{< output >}} -Untracked files: - .gitignore - 1.bak - a.bin - b.bin - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - one.bak - subdir1/ - subdir2/ - subdir3/ -{{< /output >}} - -1. A file in a specific directory is ignored in much the same way. Add the entire file path, relative to the `gitignore` file, as a new line in the file. For example, to ignore the file `file7.txt` in the directory `subdir1`, add another entry to `gitignore`, like so: - - ```file {title="testgit/.gitignore" hl_lines="2"} - file1.txt - subdir1/file7.txt - ``` - -1. Run `git status` again: - - {{< output >}} -Untracked Files: - .gitignore - 1.bak - a.bin - b.bin - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - one.bak - subdir2/ - subdir3/ -{{< /output >}} - - Here, `subdir1` is not listed because its only contents was `file7.txt.`, which is now ignored. - -### How to Add Folders to gitignore - -`gitignore` can also be used to ignore entire directories, along with any files and subdirectories in the directory. To ignore a specific directory, append a `/` symbol to the end of the directory name. - -{{< note >}} -If the `/` symbol is not added to the end of the rule, Git ignores all files and directories matching the pattern. `/` restricts the rule so it only applies to directories. -{{< /note >}} - -This example explains how to ignore the `subdir2` directory in `gitignore`. - -1. Add a new entry to `gitignore` consisting of the name of the directory to ignore: - - ```file {title="testgit/.gitignore" hl_lines="3"} - file1.txt - subdir1/file7.txt - subdir2/ - ``` - -1. Confirm the directory is now on the ignore list. Neither the directory nor the files it contains should be listed under `untracked files`: - - ```command - git status - ``` - - {{< output >}} -Untracked files: - .gitignore - 1.bak - a.bin - b.bin - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - one.bak - subdir3/ -{{< /output >}} - -### Debugging gitignore - -Git provides a debug command for determining why a file is being ignored or considered. Use the `check-ignore` command and the `-v` verbose flag. Git lists every rule that applies to the file. - -```command -git check-ignore -v subdir2/file8.txt -``` - -{{< output >}} -.gitignore:3:subdir2/ subdir2/file8.txt -{{< /output >}} - -## gitignore Syntax and Patterns - -`gitignore` is equipped with a powerful and flexible set of special operators for filtering files on a highly granular level. The `gitignore` syntax uses wildcard and exclude symbols to add multiple files or remove other files from the set of ignored files. - -This section describes the different characters comprising the `gitignore` syntax. The `check-ignore` command illustrates how the gitignore syntax affects different files. For more extensive information about the syntax, consult the [gitignore documentation](https://git-scm.com/docs/gitignore). - -### The Wildcard Symbols - -The `*` symbol matches zero or more characters, excluding only the `/` character. For example, the rule `*.bak` ignores all files with the `.bak` extension, including `1.bak` and `one.bak`. Wildcards can be used in both file and folder names. - -1. Add a new line to `.gitignore`: - - ```file {title="testgit/.gitignore" hl_lines="4"} - file1.txt - subdir1/file7.txt - subdir2/ - *.bak - ``` - -1. Use `git-status` to confirm both `.bak` files are unlisted: - - {{< output >}} -Untracked Files: - .gitignore - a.bin - b.bin - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - subdir3/ -{{< /output >}} - -1. A closely-related filter is the `?` character. This matches any single character. The rule `?.bak` matches `1.bak`, but not `one.bak`. Modify the last change to `.gitignore` to look like so: - - ```file {title="testgit/.gitignore" hl_lines="4"} - file1.txt - subdir1/file7.txt - subdir2/ - ?.bak - ``` - -1. Use `git check-ignore ` to look for `1.bak`: - - ```command - git check-ignore -v 1.bak - ``` - - {{< output >}} -.gitignore:4:?.bak 1.bak -{{< /output >}} - -1. Use `git status` again to confirm that `one.bak` is listed as untracked: - - {{< output >}} -Untracked Files: - .gitignore - a.bin - b.bin - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - one.bak - subdir3/ -{{< /output >}} - -### The Double Asterisk Symbol - -The `**` character matches any number of directories or files. This is often used to ignore certain files in a specific directory anywhere in the repository. For instance, the pattern `**/backup/*.log` matches any files ending in `.log` in any directory named `backup`. - -The `**` works slightly differently in different contexts. The pattern `**/dirname` matches all instances of the directory. The pattern `dirname/**/filename` matches files named `filename` inside `dirname` or any of its subdirectories. - -1. Add a new line to `.gitignore`: - - ```file {title="testgit/.gitignore" hl_lines="5"} - file1.txt - subdir1/file7.txt - subdir2/ - ?.bak - **/subdir3/*.log - ``` - -1. Use `git check-ignore` to look for the `files.log` file: - - ```command - git check-ignore -v subdir3/files.log - ``` - - {{< output >}} -.gitignore:5:**/subdir3/*.log subdir3/files.log -{{< /output >}} - -1. Now use `git status` to confirm that the `files.log` file's otherwise empty parent directory `subdir3` is now unlisted: - - {{< output >}} -Untracked Files: - .gitignore - a.bin - b.bin - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - one.bak -{{< /output >}} - -### The Negation Symbol - -The negation symbol removes some of the files or folders that match an earlier rule enforcing an ignored state. If the rule `*.bin` ignores all `.bin` files, then the rule `!a.bin` overrides this rule for `a.bin`. It tells Git to stop ignoring these files and move them back to the untracked state. - -{{< note >}} -Some of the negated files can be returned to the ignored state using yet another rule later in the file. So it is possible to ignore a set of files `a`, then negate subset `b` out of `a`, then ignore subset `c` from `b`. It is possible to build a long chain of nested rules using this strategy. However, this structure can be difficult to debug and should normally be avoided. -{{< /note >}} - -This example demonstrates how the rule `!a.bin` overrides the `*.bin` rule. The file `b.bin` is still ignored, but `a.bin` is listed as untracked. - -1. Add two new lines in `.gitignore`: - - ```file {title="testgit/.gitignore" hl_lines="6,7"} - file1.txt - subdir1/file7.txt - subdir2/ - ?.bak - **/subdir3/*.log - *.bin - !a.bin - ``` - -1. Use `git check-ignore` to look for `b.bin`: - - ```command - git check-ignore -v b.bin - ``` - - {{< output >}} -.gitignore:6:*.bin b.bin -{{< /output >}} - -1. Now look for `a.bin`: - - ```command - git check-ignore -v a.bin - ``` - - {{< output >}} -.gitignore:7:!a.bin a.bin -{{< /output >}} - -1. Use `git status` to confirm that `a.bin` is still untracked, but `b.bin` is unlisted: - - {{< output >}} -Untracked Files: - .gitignore - a.bin - file2.txt - file3.txt - file4.txt - file5.txt - file6.txt - one.bak -{{< /output >}} - -## The Range Symbol - -The square brackets `[]` are used to specify a numerical or alphabetical range. There are several permutations of this symbol. - -- `[0-9]` matches any single character from the range, so any number between `0` and `9`. This is the same as any single digit. -- `[01]` matches any character from the set, in this case, either `0` or `1`. -- `[!01]` matches any character except the ones in the set. -- `[a-m]` is an alphabetic range. This range includes lower case letters from `a` to `m`. - -As an example, the `gitignore` entry `file[3-5].txt` ignores `file3.txt`, `file4.txt` and `file5.txt`, but not `file2.txt` or `file6.txt`. - -1. Add `file[3-5].txt` to your `gitignore` file: - - ```file {title="testgit/.gitignore" hl_lines="8"} - file1.txt - subdir1/file7.txt - subdir2/ - ?.bak - **/subdir3/*.log - *.bin - !a.bin - file[3-5].txt - ``` - -1. Use `git status` to confirm the removal of `file3.txt`, `file4.txt`, and `file5.txt` from the list of untracked files: - - {{< output >}} -.gitignore -a.bin -file2.txt -file6.txt -one.bak -{{< /output >}} - -1. Use `git check-ignore` to look for `file3.txt`: - - ```command - git check-ignore -v file3.txt - ``` - - {{< output >}} -.gitignore:8:file[3-5].txt file3.txt -{{< /output >}} - -### The Comment Symbol - -Any entry in the `gitignore` file beginning with the `#` symbol is a comment. Comments can help organize and explain highly complicated `gitignore` files. - -Blank lines are also ignored. Developers can use them to separate the `gitignore` file into sections. - -### Combinations and Exceptions - -There are a few puzzling exceptions to the `gitignore` rules. Certain patterns are also confusing. Here are some specific cases that might cause problems. - -- For performance reasons, it is not possible to negate a file that belongs to an ignored directory. For example, if a rule ignores the `backup` directory, then Git does not acknowledge the subsequent pattern `!backup/data.log`. The `backup/data.log` file is still ignored and does not appear as an untracked file in `git status`. -- Prepending a directory separator `/` symbol to a rule indicates the rule is relative to the root directory containing the `gitignore` file. Without the `/` symbol, the rule applies everywhere in the directory. -- Patterns specifying a particular file in a certain directory are always relative to the `gitignore` file. This means the pattern `backup/debug.log` does not match the file `project/backup/debug.log`. This rule is equivalent to `/backup/debug.log`. -- Any pattern with a directory separator `/` symbol in the middle of a pattern is also relative to the `gitignore` file. -- If there is a `/` symbol at the end of a pattern, it only matches directories. Otherwise it matches both directories and files. -- The `\` symbol is an escape character. It tells Git to treat the next character as a literal character and not a special symbol. The rule `log\[05\].txt` is used to ignore `log[05].txt`. Without the escape character, the rule would ignore `log0.txt` and `log5.txt`. - -## Ignoring Files in Special Circumstances - -### Ignoring Files Locally and Globally - -The `gitignore` file is typically checked into the Git repository. This means it applies to every instance of the repository, but not to other repositories on the system. However, Git provides options to expand the rules to all repositories or only apply the rules locally. - -- **Local Repository Rules**: Rules in the `.git/info/exclude` file only apply in the local repository. This file is not checked in, so it does not apply to other copies of the repository. This is a good choice for special rules that only apply to your personal repository, including personal data or local environments. The regular `gitignore` rules still apply in this context. -- **Global gitignore Rules**: To ignore files in all repositories on a particular system, use a global `.gitignore` file. Run the following command to register the file globally with Git, then add the rules to `~/.gitignore`. - - ```command - git config --global core.excludesFile ~/.gitignore - ``` - -### Ignoring Checked-In Files - -Git does not ignore any checked in files, even if they are covered by patterns in the `gitignore` file. To ignore a checked in file, first remove it from Git. Use this command to remove and ignore the file. - -```command -git rm --cached FILENAME -``` -{{< note >}} -Conversely, it is possible to check in an ignored file using the `-f` option. Run the command `git add -f FILENAME`. After the file is checked in, Git no longer ignores it. However, this command is not recommended. It is better to create a `gitignore` pattern exempting the file, or to design the patterns so the file is never ignored in the first place. -{{< /note >}} - -## Conclusion - -Git files are normally either tracked or untracked. However, Git provides a mechanism for ignoring untracked files that are not intended for check-in. This means they do not appear in the output of commands such as `git status`, resulting in a cleaner and better organized workspace. - -Developers can ignore files and folders by adding them to the `.gitignore` file. `gitignore` is a plain text file normally located in the root directory of the repository. Powerful operators including wild card characters and exclusion characters allow users to define rules with wider or more granular scopes. For more information on how to ignore files in Git, see the [Gitignore documentation](https://git-scm.com/docs/gitignore). \ No newline at end of file diff --git a/docs/guides/development/version-control/rename-git-branch/index.md b/docs/guides/development/version-control/rename-git-branch/index.md deleted file mode 100644 index 847475acd50..00000000000 --- a/docs/guides/development/version-control/rename-git-branch/index.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -slug: rename-git-branch -title: "How to Rename a Git Branch" -title_meta: "How to Rename a Branch in Git & Why You Should" -description: 'Need to know how to rename a Git branch? This guide explains what a Git branch is, when you should rename, and how to rename a branch in Git. ✓ Click here!' -authors: ["Martin Heller"] -contributors: ["Martin Heller"] -published: 2023-04-25 -keywords: ['rename git branch','git rename branch','rename branch git','git rename local branch','git change branch name','git branch rename','git rename remote branch','rename branch','how to rename a branch in git','change branch name git'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Career Karma: How to Rename a Git Branch](https://careerkarma.com/blog/git-rename-branch/ -)' -- '[Stack Overflow: How do I rename a local Git branch?](https://stackoverflow.com/questions/6591213/how-do-i-rename-a-local-git-branch -)' -- '[TheServerSide: How to rename and change a Git branch name locally & remotely](https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/rename-Git-branch-local-remote-GitHub-GitLab-BitBucket -)' ---- - -Git is a distributed version control system that views data as a series of snapshots. This is in contrast to delta-based version control systems, such as Concurrent Version System (CVS) and Subversion. These view their data as a set of files, and the changes made to each file over time. - -In Git, once a copy of the remote repository has been checked out, work can be done on the local copy. Changes can be committed whenever needed. When ready, simply *push* these changes to the remote repository. - -## What is a Git Branch? - -Work in a *branch* in order to isolate changes made locally from changes that other team members are making on their own machines. A Git branch is basically a label for a single or series of *commits*, or changes, that are related. A commit contains a snapshot of saved code along with a link to the previous commit. The default branch name in Git is **master**, although many repositories use the name **main** instead. - -Unlike CVS or Subversion, Git encourages the free creation of development branches. The idea is to work in them, and later merge them into project, version, QA, or production branches. While working in a local branch, there is little fear of incomplete and/or untested code winding up in a production build. - -## Why Rename a Git Branch? - -It’s common to name a new Git branch with your initials (or git handle) along a date, number, or description. Once code in the branch is complete, the project committers may require that a pull request (PR) be submitted. This starts the process of having the code reviewed and merged. - -However, committers need to know what the branch is intended to accomplish, so renaming the branch to summarize its purpose may be required. The pull request provides additional space to describe the code in more detail. Different projects may have different standards for branch names submitted in pull requests. The general best practice is for the name to be descriptive of the changes made. - -Another reason to rename a Git branch would be if there’s an error in the original name. For example, in some projects the name of a branch is supposed to contain the number of the bug or issue being fixed. If the bug number is *1291* and you accidentally named the branch *bugfix-1292*, then you'd want to rename the branch to *bugfix-1291*. - -## How to Rename a Branch in Git - -### Steps to Rename the Local and Remote Branches - -Branches in a local repository can be renamed from the command line using the `git` command. However, additional steps are needed to push the change into the remote origin repository. - -{{< note >}} -This guide assumes that the remote origin repository was set when checked out. It also assumes that your terminal is in the working directory of the local repository, so that the `git` command can find the `.git` subdirectory. -{{< /note >}} - -1. Use the following command to display a list of local branches: - - ```command - git branch - ``` - - The current branch is shown with an asterisk (*): - - ```output - main - * example-branch - ``` - -1. Now use the `-r` flag to display a list of remote-tracking branches: - - ```command - git branch -r - ``` - - ```output - origin/main - origin/example-branch - ``` - -1. Create a new branch using the following command syntax `git branch `: - - ```command - git branch bugfix-1292 - ``` - -To rename this branch to `bugfix-1291`, either switch into the branch or use the long form of the `git branch -m` command. - -1. Switch into the new branch using the following command syntax `git checkout `: - - ```command - git checkout bugfix-1292 - ``` - -1. Now rename the branch to `bugfix-1291`: - - ```command - git branch -m bugfix-1291 - ``` - -Alternatively, if not already in branch `bugfix-1292`, use the two-parameter rename command syntax `git branch -m `: - -```command -git branch -m bugfix-1292 bugfix-1291 -``` - -1. When done, push the renamed branch to the remote repository: - - ```command - git push origin -u bugfix-1291 - ``` - -1. If the branch was previously pushed to the remote repository, the old branch name still exists, so delete it: - - ```command - git push origin --delete bugfix-1292 - ``` - -## Conclusion - -Git branches help isolate the changes you make to a project from changes others are making. This avoids having incomplete code released to production. There are several reasons to rename a branch, but most have to do with a project’s naming conventions and chosen best practices. You now know how to rename a local branch. This includes pushing a renamed branch to, and deleting an old branch name from a remote repository. \ No newline at end of file diff --git a/docs/guides/email/postfix/email-with-postfix-dovecot-and-mariadb-on-centos-7/index.md b/docs/guides/email/postfix/email-with-postfix-dovecot-and-mariadb-on-centos-7/index.md deleted file mode 100644 index 1ce43ac11d9..00000000000 --- a/docs/guides/email/postfix/email-with-postfix-dovecot-and-mariadb-on-centos-7/index.md +++ /dev/null @@ -1,859 +0,0 @@ ---- -slug: email-with-postfix-dovecot-and-mariadb-on-centos-7 -title: 'Email with Postfix, Dovecot and MariaDB on CentOS 7' -description: 'This guide provides you with step-by-step instructions for installing the Postfix MTA along with the Dovecot mail server and MariaDB on CentOS.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2015-03-26 -modified: 2018-10-10 -keywords: ["postfix centos 7", "dovecot centos 7", "linux mail server", "email", "centos 7"] -tags: ["centos","postfix","email","mariadb"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[Troubleshooting Problems with Postfix, Dovecot, and MySQL](/cloud/guides/troubleshooting-problems-with-postfix-dovecot-and-mysql)' - - '[Postfix Basic Configuration](http://www.postfix.org/BASIC_CONFIGURATION_README.html)' - - '[Postfix SASL Howto](http://www.postfix.org/SASL_README.html)' - - '[Dovecot Wiki](https://wiki2.dovecot.org/)' ---- - -In this guide, you'll learn how to set up a secure virtual user mail server with Postfix, Dovecot, and MariaDB (a drop-in replacement for MySQL) on CentOS 7. We'll explain how to create new user mailboxes and send or receive email to and from configured domains. - -![Email with Postfix, Dovecot and MariaDB on CentOS 7](Email_with_Postfix_Dovecot_and_MariaDB_on_CentOS_7_smg.jpg) - -For a different Linux distribution or different mail server, review our [email tutorials](/cloud/guides/email). - -{{% content "email-warning-shortguide" %}} - -## Before You Begin - -1. Set up the Linode as specified in the [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) and [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide. - -1. Verify that the iptables [firewall](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-firewall) is not blocking any of the standard mail ports (`25`, `465`, `587`, `110`, `995`, `143`, and `993`). If using a different form of firewall, confirm that it is not blocking any of the needed ports. - -1. Review the concepts in the [Running a Mail Server](/cloud/guides/running-a-mail-server) guide. - -## Configure DNS - -When you're ready to update the DNS and start sending mail to the server, edit the domain's MX record so that it points to the Linode's domain or IP address, similar to the example below: - -{{< output >}} -example.com A 10 12.34.56.78 -example.com MX 10 example.com -mail.example.com MX 10 example.com -{{< /output >}} - -Make sure that the MX record is changed for all domains and subdomains that might receive email. If setting up a brand new domain, these steps can be performed prior to configuring the mail server. When using Linode's [DNS Manager](https://techdocs.akamai.com/cloud-computing/docs/dns-manager), create an MX record that points to the desired domain or subdomain, and then create an A record for that domain or subdomain, which points to the correct IP address. - -## Update Hosts File - -Verify that the `hosts` file contains a line for the Linode's public IP address and is associated with the **Fully Qualified Domain Name** (FQDN). In the example below, `192.0.2.0` is the public IP address, `hostname` is the local hostname, and `hostname.example.com` is the FQDN. - -{{< file "/etc/hosts" h >}} -127.0.0.1 localhost.localdomain localhost -192.0.2.0 hostname.example.com hostname - -{{< /file >}} - -## Install SSL Certificate - -You will need to install a SSL certificate on your mail server prior to completing the [Dovecot](#dovecot) configuration steps. The SSL certificate will authenticate the identity of the mail server to users and encrypt the transmitted data between the user's mail client and the mail server. - -{{% content "certbot-shortguide-centos" %}} - -Make a note of the certificate and key locations on the Linode. You will need the path to each during the [Dovecot](#dovecot) configuration steps. - -## Install Required Packages - -1. Install any outstanding package updates: - - yum update - -1. Install the required packages: - - yum install postfix dovecot mariadb-server dovecot-mysql - - This installs the Postfix mail server, the MariaDB database server, the Dovecot IMAP and POP daemons, and several supporting packages that provide services related to authentication. - -### Versions - -This guide uses the following package versions: - -* Postfix 2.10.1 -* Dovecot 2.2.10 -* MariaDB 5.5.60 - -## MariaDB - -In this section you will set up a MariaDB database to store virtual domains, users and passwords. Dovecot and Postfix require this data. - -### Creating the Database and Tables - -Follow the steps below to create the database tables for virtual users, domains and aliases: - -1. Ensure the MariaDb server is running and enabled to start automatically on reboot: - - sudo systemctl start mariadb - sudo systemctl enable mariadb - -1. Use the [*mysql_secure_installation*](https://mariadb.com/kb/en/library/mysql_secure_installation/) tool to configure additional security options. You will be given the choice to change the MariaDB root password, remove anonymous user accounts, disable root logins outside of localhost, and remove test databases. It is recommended that you answer `yes` to these options. You can read more about the script in the [MariaDB Knowledge Base](https://mariadb.com/kb/en/mariadb/mysql_secure_installation/). - - sudo mysql_secure_installation - -1. Create a new database: - - sudo mysqladmin -u root -p create mailserver - -1. Log in to MySQL: - - sudo mysql -u root -p - -1. Create the MySQL user and grant the new user permissions over the database. Replace `mailuserpass` with a secure password: - - GRANT SELECT ON mailserver.* TO 'mailuser'@'127.0.0.1' IDENTIFIED BY 'mailuserpass'; - -1. Flush the MySQL privileges to apply the change: - - FLUSH PRIVILEGES; - -1. Switch to the new `mailsever` database: - - USE mailserver; - -1. Create a table for the domains that will receive mail on the Linode: - - CREATE TABLE `virtual_domains` ( - `id` int(11) NOT NULL auto_increment, - `name` varchar(50) NOT NULL, - PRIMARY KEY (`id`) - ) ENGINE=InnoDB DEFAULT CHARSET=utf8; - -1. Create a table for all of the email addresses and passwords: - - CREATE TABLE `virtual_users` ( - `id` int(11) NOT NULL auto_increment, - `domain_id` int(11) NOT NULL, - `password` varchar(106) NOT NULL, - `email` varchar(100) NOT NULL, - PRIMARY KEY (`id`), - UNIQUE KEY `email` (`email`), - FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE - ) ENGINE=InnoDB DEFAULT CHARSET=utf8; - -1. Create a table for the email aliases: - - CREATE TABLE `virtual_aliases` ( - `id` int(11) NOT NULL auto_increment, - `domain_id` int(11) NOT NULL, - `source` varchar(100) NOT NULL, - `destination` varchar(100) NOT NULL, - PRIMARY KEY (`id`), - FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE - ) ENGINE=InnoDB DEFAULT CHARSET=utf8; - -### Adding Data - -You can now add data to the database and tables that were created in the previous section. - -1. Add the domains to the `virtual_domains` table. Replace the values for `example.com` and `hostname` with your own settings: - - INSERT INTO `mailserver`.`virtual_domains` - (`id` ,`name`) - VALUES - ('1', 'example.com'), - ('2', 'hostname.example.com'), - ('3', 'hostname'), - ('4', 'localhost.example.com'); - - {{< note respectIndent=false >}} -Note which `id` corresponds to which domain, the `id` value is necessary for the next two steps. -{{< /note >}} - -1. Add email addresses to the `virtual_users` table. The `domain_id` value references the `virtual_domain` table's `id` value. Replace the email address values with the addresses that you wish to configure on the mailserver. Replace the `password` values with strong passwords. - - INSERT INTO `mailserver`.`virtual_users` - (`id`, `domain_id`, `password` , `email`) - VALUES - ('1', '1', ENCRYPT('password', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email1@example.com'), - ('2', '1', ENCRYPT('password', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email2@example.com'); - -1. An email alias will forward all email from one email address to another. To set up an email alias, add it to the `virtual_aliases` table: - - INSERT INTO `mailserver`.`virtual_aliases` - (`id`, `domain_id`, `source`, `destination`) - VALUES - ('1', '1', 'alias@example.com', 'email1@example.com'); - -### Testing - -In the previous section, data was added to the MySQL `mailserver` database. The steps below will test that the data has been stored and can be retrieved. - -1. Log in to MySQL: - - sudo mysql -u root -p - -1. Check the contents of the `virtual_domains` table: - - SELECT * FROM mailserver.virtual_domains; - -1. Verify that the output displays the domains you add to the `virtual_domains` table: - - {{< output >}} -+----+-----------------------+ -| id | name | -+----+-----------------------+ -| 1 | example.com | -| 2 | hostname.example.com | -| 3 | hostname | -| 4 | localhost.example.com | -+----+-----------------------+ -4 rows in set (0.00 sec) -{{}} - -1. Check the `virtual_users` table: - - SELECT * FROM mailserver.virtual_users; - -1. Verify that the output displays the email addresses you added to the `virutal_users` table. Your hashed passwords will appear longer than they are displayed below: - - {{< output >}} -+----+-----------+-------------------------------------+--------------------+ -| id | domain_id | password | email | -+----+-----------+-------------------------------------+--------------------+ -| 1 | 1 | $6$574ef443973a5529c20616ab7c6828f7 | email1@example.com | -| 2 | 1 | $6$030fa94bcfc6554023a9aad90a8c9ca1 | email2@example.com | -+----+-----------+-------------------------------------+--------------------+ -2 rows in set (0.01 sec) -{{}} - -1. Check the `virtual_aliases` table: - - SELECT * FROM mailserver.virtual_aliases; - -1. Verify that the output displays the aliases you added to the `virtual_aliases` table: - - {{< output >}} -+----+-----------+-------------------+--------------------+ -| id | domain_id | source | destination | -+----+-----------+-------------------+--------------------+ -| 1 | 1 | alias@example.com | email1@example.com | -+----+-----------+-------------------+--------------------+ -1 row in set (0.00 sec) -{{}} - -1. If all the desired data displays as expected, exit MySQL: - - exit - -## Postfix - -Postfix is a *Mail Transfer Agent* (MTA) that relays mail between the Linode and the internet. It is highly configurable, allowing for great flexibility. This guide maintains many of Posfix's default configuration values. - -### Configuration File Settings - -The `main.cf` file is the primary configuration file used by Postfix. - -1. Make a copy of the default Postfix configuration file in case you need to revert to the default configuration: - - sudo cp /etc/postfix/main.cf /etc/postfix/main.cf.orig - -1. Edit the `/etc/postfix/main.cf` file to match the example configurations. Replace occurrences of `example.com` with your domain name: - - {{< file "/etc/postfix/main.cf" >}} -# See /usr/share/postfix/main.cf.dist for a commented, more complete version - -# Debian specific: Specifying a file name will cause the first -# line of that file to be used as the name. The Debian default -# is /etc/mailname. -#myorigin = /etc/mailname - -smtpd_banner = $myhostname ESMTP $mail_name (CentOS) -biff = no - -# appending .domain is the MUA's job. -append_dot_mydomain = no - -# Uncomment the next line to generate "delayed mail" warnings -#delay_warning_time = 4h - -readme_directory = no - -# TLS parameters -smtpd_tls_cert_file=/etc/letsencrypt/live/example.com/fullchain.pem -smtpd_tls_key_file=/etc/letsencrypt/live/example.com/privkey.pem -smtpd_use_tls=yes -smtpd_tls_auth_only = yes -smtp_tls_security_level = may -smtpd_tls_security_level = may -smtpd_sasl_security_options = noanonymous, noplaintext -smtpd_sasl_tls_security_options = noanonymous - -# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for -# information on enabling SSL in the smtp client. -smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination -myhostname = example.com -alias_maps = hash:/etc/aliases -alias_database = hash:/etc/aliases -mydomain = example.com -myorigin = $mydomain -mydestination = localhost, localhost.$mydomain -relayhost = -mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 -mailbox_size_limit = 0 -recipient_delimiter = + -inet_interfaces = all -inet_protocols = all - -# Handing off local delivery to Dovecot's LMTP, and telling it where to store mail -virtual_transport = lmtp:unix:private/dovecot-lmtp - -# Virtual domains, users, and aliases -virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf -virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf -virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf, - mysql:/etc/postfix/mysql-virtual-email2email.cf - -{{< /file >}} - -1. The `main.cf` file declares the location of `virtual_mailbox_domains`, `virtual_mailbox_maps`, `virtual_alias_maps`, and `mysql-virtual-email2email` files. These files contain the connection information for the MySQL lookup tables created in the [MariaDB](#mariadb) section of this guide. Postfix will use this data to identify all domains, corresponding mailboxes, and valid users. - - Create the file for `virtual_mailbox_domains`. Replace the value for `password` with your database user's password. This password was created in the [Creating the Database and Tables](#creating-the-database-and-tables) section. If you used a different name for your database `user` and `dbname` replace those with your own values: - - {{< file "/etc/postfix/mysql-virtual-mailbox-domains.cf" >}} -user = mailuser -password = mailuserpass -hosts = 127.0.0.1 -dbname = mailserver -query = SELECT 1 FROM virtual_domains WHERE name='%s' - -{{< /file >}} - -1. Create the `/etc/postfix/mysql-virtual-mailbox-maps.cf` file, and enter the following values. Use the database user's password and make any other changes as needed: - - {{< file "/etc/postfix/mysql-virtual-mailbox-maps.cf" >}} -user = mailuser -password = mailuserpass -hosts = 127.0.0.1 -dbname = mailserver -query = SELECT 1 FROM virtual_users WHERE email='%s' - -{{< /file >}} - -1. Create the `/etc/postfix/mysql-virtual-alias-maps.cf` file and enter the following values. Use the database user's password and make any other changes as needed: - - {{< file "/etc/postfix/mysql-virtual-alias-maps.cf" >}} -user = mailuser -password = mailuserpass -hosts = 127.0.0.1 -dbname = mailserver -query = SELECT destination FROM virtual_aliases WHERE source='%s' - -{{< /file >}} - -1. Create the `/etc/postfix/mysql-virtual-email2email.cf` file and enter the following values. Use the database user's password and make any other changes as needed: - - {{< file "/etc/postfix/mysql-virtual-email2email.cf" >}} -user = mailuser -password = mailuserpass -hosts = 127.0.0.1 -dbname = mailserver -query = SELECT email FROM virtual_users WHERE email='%s' - -{{< /file >}} - -1. Restart Postfix: - - sudo systemctl restart postfix - -1. The `postmap` command creates or queries Postfix's lookup tables, or updates an existing one. Enter the following command to ensure that Postfix can query the `virtual_domains` table. Replace `example.com` with the first `name` value. The command should return `1` if it is successful: - - sudo postmap -q example.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf - -1. Test Postfix to verify that it can retrieve the first email address from the MySQL table `virtual_users`. Replace `email1@example.com` with the first email address added to the table. You should receive `1` as the output: - - sudo postmap -q email1@example.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf - - Verify that Postfix can retrieve the first email address from the MySQL table `virtual_users` using the database values entered in the `mysql-virtual-mailbox-maps.cf` file. Replace `email1@example.com` with the first email address added to the table. You should receive `1` as the output: - - sudo postmap -q alias@example.com mysql:/etc/postfix/mysql-virtual-email2email.cf - -1. Test Postfix to verify that it can query the `virtual_aliases` table. Replace `alias@example.com` with the first `source` value created in the table. The command should return the `destination` value for the row: - - sudo postmap -q alias@example.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf - -### Master Program Settings - -Postfix's master program starts and monitors all of Postfix's processes. The configuration file `master.cf` lists all programs and information on how they should be started. - -1. Make a copy of the `/etc/postfix/master.cf` file: - - sudo cp /etc/postfix/master.cf /etc/postfix/master.cf.orig - -1. Edit `/etc/postfix/master.cf` to contain the values in the excerpt example. The rest of the file can remain unchanged: - - {{< file "/etc/postfix/master.cf" >}} -# -# Postfix master process configuration file. For details on the format -# of the file, see the master(5) manual page (command: "man 5 master" or -# on-line: http://www.postfix.org/master.5.html). -# -# Do not forget to execute "postfix reload" after editing this file. -# -# ========================================================================== -# service type private unpriv chroot wakeup maxproc command + args -# (yes) (yes) (yes) (never) (100) -# ========================================================================== -smtp inet n - n - - smtpd -#smtp inet n - - - 1 postscreen -#smtpd pass - - - - - smtpd -#dnsblog unix - - - - 0 dnsblog -#tlsproxy unix - - - - 0 tlsproxy -submission inet n - - - - smtpd - -o syslog_name=postfix/submission - -o smtpd_tls_security_level=encrypt - -o smtpd_sasl_auth_enable=yes - -o smtpd_sasl_type=dovecot - -o smtpd_sasl_path=private/auth - -o smtpd_reject_unlisted_recipient=no - -o smtpd_client_restrictions=permit_sasl_authenticated,reject - -o milter_macro_daemon_name=ORIGINATING -smtps inet n - - - - smtpd - -o syslog_name=postfix/smtps - -o smtpd_tls_wrappermode=yes - -o smtpd_sasl_auth_enable=yes - -o smtpd_sasl_type=dovecot - -o smtpd_sasl_path=private/auth - -o smtpd_client_restrictions=permit_sasl_authenticated,reject - -o milter_macro_daemon_name=ORIGINATING - ... - -{{< /file >}} - -1. Change the permissions of the `/etc/postfix` directory to restrict permissions to allow only its owner and the corresponding group: - - sudo chmod -R o-rwx /etc/postfix - -1. Restart Postfix: - - sudo systemctl restart postfix - - If you get the `fatal: 0.0.0.0:smtps: Servname not supported for ai_socktype` error after restarting Postfix, add following service entries to `/etc/services`: - - {{< file "/etc/services">}} -smtps 465/tcp # Secure SMTP -smtps 465/udp # Secure SMTP -{{< /file >}} - -## Dovecot - -Dovecot is the *Mail Delivery Agent* (MDA) which is passed messages from Postfix and delivers them to a virtual mailbox. In this section, configure Dovecot to force users to use SSL when they connect so that their passwords are never sent to the server in plain text. - -1. Copy all of the configuration files so you can easily revert back to them if needed: - - sudo cp /etc/dovecot/dovecot.conf /etc/dovecot/dovecot.conf.orig - sudo cp /etc/dovecot/conf.d/10-mail.conf /etc/dovecot/conf.d/10-mail.conf.orig - sudo cp /etc/dovecot/conf.d/10-auth.conf /etc/dovecot/conf.d/10-auth.conf.orig - sudo cp /etc/dovecot/conf.d/auth-sql.conf.ext /etc/dovecot/conf.d/auth-sql.conf.ext.orig - sudo cp /etc/dovecot/conf.d/10-master.conf /etc/dovecot/conf.d/10-master.conf.orig - sudo cp /etc/dovecot/conf.d/10-ssl.conf /etc/dovecot/conf.d/10-ssl.conf.orig - -1. Edit the `/etc/dovecot/dovecot.conf` file. Uncomment `protocols = imap pop3 lmtp` in the `# Protocols we want to be serving.` section of the file: - - {{< file "/etc/dovecot/dovecot.conf" >}} -## Dovecot configuration file -... -# Protocols we want to be serving. -protocols = imap pop3 lmtp - -{{< /file >}} - -1. Edit the `/etc/dovecot/conf.d/10-mail.conf` file. This file controls how Dovecot interacts with the server's file system to store and retrieve messages. - - Modify the following variables within the configuration file: - - {{< file "/etc/dovecot/conf.d/10-mail.conf" >}} -... -mail_location = maildir:/var/mail/vhosts/%d/%n -... -mail_privileged_group = mail -... -{{< /file >}} - -1. Create the `/var/mail/vhosts/` directory and a subdirectory for your domain. Replace `example.com` with your domain name: - - sudo mkdir -p /var/mail/vhosts/example.com - - This directory will serve as storage for mail sent to your domain. - -1. Create the `vmail` group with ID `5000`. Add a new user `vmail` to the `vmail` group. This system user will read mail from the server. - - sudo groupadd -g 5000 vmail - sudo useradd -g vmail -u 5000 vmail -d /var/mail/ - -1. Change the owner of the `/var/mail/` folder and its contents to belong to `vmail`: - - sudo chown -R vmail:vmail /var/mail/ - -1. Edit the user authentication file, located in `/etc/dovecot/conf.d/10-auth.conf`. Uncomment the following variables and replace with the file excerpt's example values: - - {{< file "/etc/dovecot/conf.d/10-auth.conf" >}} -... -disable_plaintext_auth = yes -... -auth_mechanisms = plain login -... -!include auth-system.conf.ext -... -!include auth-sql.conf.ext -... - -{{< /file >}} - -1. Edit the `/etc/dovecot/conf.d/auth-sql.conf.ext` file with authentication and storage information. Ensure your file contains the following lines and that they are uncommented: - - {{< file "/etc/dovecot/conf.d/auth-sql.conf.ext" >}} -... -passdb { - driver = sql - args = /etc/dovecot/dovecot-sql.conf.ext -} -... -userdb { - driver = static - args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n -} -... - -{{< /file >}} - -1. Create the `/etc/dovecot/dovecot-sql.conf.ext` file and update it with your MySQL connection information. Add the following variables and replace the values with the excerpt example. Replace `dbname`, `user` and `password` with your own MySQL database values: - - {{< file "/etc/dovecot/dovecot-sql.conf.ext" >}} -driver = mysql -connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mailuserpass -default_pass_scheme = SHA512-CRYPT -password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; -{{< /file >}} - - The `password_query` variable uses email addresses listed in the `virtual_users` table as the username credential for an email account. - - To use an alias as the username: - - 1. Add the alias as the `source` and `destination` email address to the `virtual_aliases` table. - 1. Change the `/etc/dovecot/dovecot-sql.conf.ext` file's `password_query` value to `password_query = SELECT email as user, password FROM virtual_users WHERE email=(SELECT destination FROM virtual_aliases WHERE source = '%u');` - -1. Change the owner and group of the `/etc/dovecot/` directory to `vmail` and `dovecot`: - - sudo chown -R vmail:dovecot /etc/dovecot - -1. Change the permissions on the `/etc/dovecot/` directory to be recursively read, write, and execute for the owner of the directory: - - sudo chmod -R o-rwx /etc/dovecot - -1. Edit the service settings file `/etc/dovecot/conf.d/10-master.conf`: - - {{< note respectIndent=false >}} -When editing the file, be careful not to remove any opening or closing curly braces. If there's a syntax error, Dovecot will crash silently. You can check `/var/log/upstart/dovecot.log` to debug the error. -{{< /note >}} - - Disable unencrypted IMAP and POP3 by setting the protocols' ports to `0`. Uncomment the `port` and `ssl` variables: - - {{< file "/etc/dovecot/conf.d/10-master.conf" >}} -... -service imap-login { - inet_listener imap { - port = 0 - } - inet_listener imaps { - port = 993 - ssl = yes - } - ... -} -... -service pop3-login { - inet_listener pop3 { - port = 0 - } - inet_listener pop3s { - port = 995 - ssl = yes - } -} -... -{{< /file >}} - - Find the `service lmtp` section of the file and use the configuration shown below: - - {{< file "/etc/dovecot/conf.d/10-master.conf" >}} -... -service lmtp { - unix_listener /var/spool/postfix/private/dovecot-lmtp { - #mode = 0666i - mode = 0600 - user = postfix - group = postfix - } -... -} -{{< /file >}} - - - Locate `service auth` and configure it as shown below: - - {{< file "/etc/dovecot/conf.d/10-master.conf" >}} -... -service auth { - ... - unix_listener /var/spool/postfix/private/auth { - mode = 0660 - user = postfix - group = postfix - } - - unix_listener auth-userdb { - mode = 0600 - user = vmail - } -... - user = dovecot -} -... - -{{< /file >}} - - - In the `service auth-worker` section, uncomment the `user` line and set it to `vmail`: - - {{< file "/etc/dovecot/conf.d/10-master.conf" >}} -... -service auth-worker { - ... - user = vmail -} - -{{< /file >}} - - - Save the changes to the `/etc/dovecot/conf.d/10-master.conf` file. - -1. Edit `/etc/dovecot/conf.d/10-ssl.conf` file to require SSL and to add the location of your domain's SSL certificate and key. Replace `example.com` with your domain: - - {{< file "/etc/dovecot/conf.d/10-ssl.conf" >}} -... -# SSL/TLS support: yes, no, required. -ssl = required -... -ssl_cert = }} - -1. Restart Dovecot to enable all configurations: - - sudo systemctl restart dovecot - -## Test Your Configuration -{{< note >}} -Given the possibility of hosting a large number of virtual domains on a single mail system, the username portion of an email address (i.e. before the `@` sign) is not sufficient to authenticate to the mail server. When email users authenticate to the server, they must supply their email clients with the *entire* email address created above as their username. -{{< /note >}} - -1. Prior to accessing any newly-created email account, a test message needs to be sent to create that user's mailbox (replace `email1@example.com` with an address that you have configured in your database): - - yum install mailx - mail email1@example.com - - Enter a subject and message (optional), then press `Ctrl+D` to complete and send the message. - - -1. After the test mail is sent, check `/var/log/maillog` to make sure the mail was delivered. - - tail /var/log/maillog - - At the end of the file, you should see something similar to the following: - - {{< file "/var/log/maillog" >}} -Mar 18 17:18:47 localhost postfix/cleanup[3427]: B624062FA: message-id=<20150318171847.B624062FA@example.com> -Mar 18 17:18:47 localhost postfix/qmgr[3410]: B624062FA: from=, size=515, nrcpt=1 (queue active) -Mar 18 17:18:47 localhost postfix/pipe[3435]: B624062FA: to=, relay=dovecot, delay=0.14, delays=0.04/0.01/0/0.09, dsn=2.0.0, $ -Mar 18 17:18:47 localhost postfix/qmgr[3410]: B624062FA: removed - -{{< /file >}} - -1. Now you can test to see what the users of your email server would see with their email clients. To test the `email1@example.com` mailbox, navigate to the mailbox directory `/var/mail/vhosts/example.com/email1` and issue the following command: - - find - -1. You should see output similar to the following: - - . - ./dovecot-uidvalidity - ./new - ./maildirfolder - ./dovecot.index.cache - ./cur - ./cur/1539202420.M440788P27259.mail,S=450,W=465:2,S - ./dovecot.index.log - ./dovecot-uidlist - ./dovecot-uidvalidity.5bbe5d50 - ./tmp - -1. Test the mailbox by using a simple mail client. For this test, using [**mutt**](http://www.mutt.org/) is recommended. If it is not installed by default, install it with `yum install mutt`, then run: - - sudo mutt -f . - - You may be prompted to create the root mailbox. This is not required. - -1. If there is an email in the inbox, Postfix, Dovecot, and MySQL have been successfully configured! To quit mutt press `q`. - - ![Your mailbox is working!](postfixcentos-mutt.png) - -## Email Client - -You can set up an email client to connect to your mail server. Many clients detect server settings automatically. Manual configuration requires the following parameters: - -- **Username:** The full email address, including the `@example.com` part. -- **Password:** The password that was entered for the email address in the `virtual_users` table of the `mailuser` database. -- **Server name:** The incoming and outgoing server names must be a domain that resolves to the Linode. -- **SSL:** Incoming and outgoing servers require authentication and SSL encryption. -- **Ports:** Use Port `993` for secure IMAP, Port `995` for secure POP3, and Port `587` with SSL for SMTP. - -See [Install SquirrelMail on Ubuntu 16.04](/cloud/guides/install-squirrelmail-on-ubuntu-16-04-or-debian-8) for details on installing an email client. - -## Adding New Domains, Email Addresses, and Aliases - -To add new domains, email addresses, and aliases to the mailserver you will need to update the corresponding MySQL tables created in the [MySQL](#mysql) section of this guide. - -### Domains - -1. To add a new domain, [connect to your Linode via SSH](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#connect-to-the-instance). - -1. Log in to the MySQL server: - - sudo mysql -u root - -1. Enter the root MySQL password when prompted. - -1. View the contents of the table before adding new entries. If you did not use `virtual_domains` as the name of your domain table, replace the value: - - SELECT * FROM mailserver.virtual_domains; - -1. The output should resemble the following: - - {{< output >}} - +----+-----------------------+ - | id | name | - +----+-----------------------+ - | 1 | example.com | - | 2 | hostname.example.com | - | 3 | hostname | - | 4 | localhost.example.com | - +----+-----------------------+ -{{}} - -1. Add a new domain to the table. Replace `newdomain.com` with the desired domain name: - - INSERT INTO `mailserver`.`virtual_domains` - (`name`) - VALUES - ('newdomain.com'); - -1. Verify that the new domain has been added. The output should display the new domain name. - - SELECT * FROM mailserver.virtual_domains; - -1. Exit MySQL: - - quit - -### Email Addresses - -1. Log in to the MySQL server: - - sudo mysql -u root - - When prompted enter the MySQL password. - -1. Verify the contents of the user table. Replace `virtual_users` with your table name: - - SELECT * FROM mailserver.virtual_users; - - The output should resemble the following: - - {{< output >}} -+----+-----------+-------------------------------------+--------------------+ -| id | domain_id | password | email | -+----+-----------+-------------------------------------+--------------------+ -| 1 | 1 | $6$574ef443973a5529c20616ab7c6828f7 | email1@example.com | -| 2 | 1 | $6$030fa94bcfc6554023a9aad90a8c9ca1 | email2@example.com | -+----+-----------+-------------------------------------+--------------------+ -2 rows in set (0.01 sec) -{{}} - -1. Add a new email address to the existing table. Replace `newpassword` with the user's password, and `email3@newdomain.com` with the user's email address: - - INSERT INTO `mailserver`.`virtual_users` - (`domain_id`, `password` , `email`) - VALUES - ('5', ENCRYPT('newpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))) , 'email3@newdomain.com'); - - {{< note respectIndent=false >}} -The `domain_id` should correspond to the `id` value of the domain in the `virtual_domains` table. In the example, we are creating an email address for `newdomain.com` added in the previous section. -{{< /note >}} - -1. Verify that the new email address has been added. The new email address should be displayed in the output: - - SELECT * FROM mailserver.virtual_users; - -1. Exit MySQL: - - quit - -### Aliases - -1. Log in to the MySQL server: - - sudo mysql -u root - - When prompted enter the MySQL password. - -1. Verify the contents of the user table. Replace `virtual_users` with your table name: - - SELECT * FROM mailserver.virtual_aliases; - - The output should resemble the following: - - {{< output>}} -+----+-----------+-------------------+--------------------+ -| id | domain_id | source | destination | -+----+-----------+-------------------+--------------------+ -| 1 | 1 | alias@example.com | email1@example.com | -+----+-----------+-------------------+--------------------+ -1 row in set (0.00 sec) -{{}} - -1. Add a new alias. Replace `alias@newdomain.com` with the address to forward email from, and `email1@gmail.com` with the address that you want to forward the mail to. The `alias@newdomain.com` needs to be an email address that already exists on the mail server: - - INSERT INTO `mailserver`.`virtual_aliases` - (`domain_id`, `source`, `destination`) - VALUES - ('5', 'alias@newdomain.com', 'myemail@gmail.com'); - - {{< note respectIndent=false >}} -The `domain_id` should correspond to the `id` value of the domain in the `virtual_domains` table. In the example, we are creating an email address for `newdomain.com` added in the previous section. -{{< /note >}} - - You can create a "catch-all" alias which will forward all emails sent to the matching domain that does not have matching aliases or users. Replace `@newdomain.com` with your domain. This value is the source of the alias. - - INSERT INTO `mailserver`.`virtual_aliases` - (`domain_id`, `source`, `destination`) - VALUES - ('5', '@newdomain.com', 'myemail@gmail.com'); - -1. Verify that the new alias has been added. The new alias will be displayed in the output: - - SELECT * FROM mailserver.virtual_aliases; - -1. Exit MySQL: - - quit diff --git a/docs/guides/game-servers/garrys-mod-server-on-centos-7/index.md b/docs/guides/game-servers/garrys-mod-server-on-centos-7/index.md deleted file mode 100644 index 44b9946c285..00000000000 --- a/docs/guides/game-servers/garrys-mod-server-on-centos-7/index.md +++ /dev/null @@ -1,174 +0,0 @@ ---- -slug: garrys-mod-server-on-centos-7 -title: "Garry's Mod on CentOS 7" -description: "This guide provides you with step-by-step instructions for deploying a Garry's Mod Server, which enables complete control of a video game engine, on CentOS 7." -authors: ["Julian Meyer"] -contributors: ["Julian Meyer"] -published: 2015-01-21 -modified: 2019-02-01 -keywords: ["garry's mod", "centos", "centos 7"] -tags: ["centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: -- '[SRCDS](http://www.srcds.com/)' -- '[GMod Forums](http://facepunch.com/forum.php)' -- '[GMod Wiki](http://wiki.garrysmod.com/page/Main_Page)' -dedicated_cpu_link: true ---- - -[Garry's Mod](http://www.garrysmod.com/) enables complete control and modification of the video game engine, Source Engine. With Garry's Mod, you can create almost any game you want. Setting up a Garry's Mod server is a great way to play with friends over the internet while maintaining control over the server. - -This guide shows how to create, maintain, and secure a Garry's Mod server. - -## Before You Begin - -1. You will need a [Steam](http://store.steampowered.com) account and a copy of [Garry's Mod](http://store.steampowered.com/app/4000/). - -2. Complete our guide: [Install SteamCMD for a Steam Game Server](/cloud/guides/install-steamcmd-for-a-steam-game-server). This will get SteamCMD installed and running on your Linode and this guide will pick up where the SteamCMD page leaves off. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Prerequisites for Garry's Mod - -From the SteamCMD guide, two additional steps are needed specifically for Gmod. - -1. Add a firewall rule. This command assumes that you have **only** the firewalld rules in place from the SteamCMD guide. This inserts a rule for port 10999 after the pre-existing rules for SteamCMD. - - sudo firewall-cmd --zone=public --add-port=27000 27030/udp --permanent - -2. Install an additional 32-bit package: - - sudo yum install ncurses-libs.i686 - -## Install Garry's Mod - -1. Be sure you are in the directory `~/Steam`, then access the `Steam>` prompt. - - cd ~/Steam && ./steamcmd.sh - -2. From the SteamCMD prompt, login anonymously: - - login anonymous - - Or log in with your Steam username: - - login example_user - -3. Install Gmod to the `Steam` user's home directory: - - force_install_dir ./gmod - app_update 4020 validate - - This can take some time. If the download looks as if it has frozen, be patient. Once the download is complete, you should see this output: - - Success! App '4020' fully installed. - - Steam> - -4. Quit SteamCMD: - - quit - - {{< note respectIndent=false >}} -To update Gmod, run the above 4 commands again. -{{< /note >}} - -## Configure Garry's Mod - -This section configures different aspects of the server, including gamemode and workshop addons. - -### Startup Script - -1. Create a startup script for Gmod with the following contents: - - {{< file "~/startgmod.sh" >}} -#!/bin/sh - -cd ./Steam/gmod -screen -S "Garry's Mod Server" ./srcds_run -game garrysmod +maxplayers 20 +map gm_flatgrass - -{{< /file >}} - - - When run, the script will change directories to `~/Steam/gmod` and execute Garry's Mod in a [Screen](/cloud/guides/using-gnu-screen-to-manage-persistent-terminal-sessions) session. The `srcds_run` binary can take many more arguments which you can see at [Valve's Developer wiki](https://developer.valvesoftware.com/wiki/Command_Line_Options#Source_Dedicated_Server). - -2. Make the script executable: - - chmod +x ~/startgmod.sh - -### Update Script - -The Garry's Mod wiki has instructions to use a script to update Gmod rather than manually through SteamCMD. See under [*Installing Garry's Mod*](http://wiki.garrysmod.com/page/Linux_Dedicated_Server_Hosting). - -### Automatic Startup After Server Reboots - -This will automatically restart Garry's Mod when your server reboots. - -1. Open Crontab (this will open in `vim`): - - crontab -e - -2. Enter the startup command. **Be sure that the binary flags are the same as in your startup script.** - - @reboot /home/steam/Steam/gmod/srcds_run -game garrysmod +maxplayers 20 +map gm_flatgrass - -3. To exit `vim`, press **Esc**, then type **:x** and hit **Enter**. - -### Server Config File - -The default `server.cfg` file is blank, and any configuration options you want to specify for the server must be added. This is optional, but below is a sane starting point. - -{{< file "~/Steam/gmod/garrysmod/cfg/server.cfg" >}} -hostname "server_hostname" -sv_password "server_password" -sv_timeout 60 -rcon_password "rcon_password" -mp_autoteambalance 1 -mp_limitteams 1 -writeid -writeip - -{{< /file >}} - - -### Workshop Addons - -1. Create a collection of addons you want to install on your server at [Garry's Mod Collections](http://steamcommunity.com/workshop/browse/?section=collections&appid=4000&p=3). You will need to be logged in to Steam. - -2. Note the collection ID. It is located at the end of the url, denoted by the 'X's here: - - http://steamcommunity.com/sharedfiles/filedetails/?id=XXXXXXXXX - -3. Acquire a Steam API key from the [Steam API Keys](http://steamcommunity.com/dev/apikey) page. Note the key. - -4. Paste the Steam API key and Workshop Collection ID into your startup script. For example: - - ./srcds_run +maxplayers 20 +gamemode terrortown +map cs_office -authkey YOURKEYGOESHERE +host_workshop_collection 157384458 - - This sets the game for the Trouble in Terrorist Town (TTT) gamemode, but can be changed to your desired gamemode, if you have it downloaded. - - For more info on Workshop Addons, see [Workshop for Dedicated Servers](http://wiki.garrysmod.com/page/Workshop_for_Dedicated_Servers) in the Garry's Mod wiki. - -## Using the Server - -1. Now that your server is installed and configured, it can be launched by running the `startgmod.sh` script from your `steam` user's home directory. - - cd ~/ && ./startgmod.sh - - {{< note type="alert" respectIndent=false >}} -From this point, do not press the **Control+C** keys while in the console unless you want to stop Gmod. -{{< /note >}} - -2. To detach from the screen session running the server console, press these two key combinations in succession: - - **Control+A** - **Control+D** - -3. To bring the console back, type the following command: - - screen -r - -4. To stop the server, bring back the Gmod console and press **CONTROL + C**. diff --git a/docs/guides/game-servers/installing-amp-game-server-management-panel/index.md b/docs/guides/game-servers/installing-amp-game-server-management-panel/index.md deleted file mode 100644 index e4d4f14f39a..00000000000 --- a/docs/guides/game-servers/installing-amp-game-server-management-panel/index.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -slug: installing-amp-game-server-management-panel -title: Installing AMP for Game Server Management -description: "AMP is a game server management panel. As the successor to McMyAdmin, it provides a web interface to install games, mods, and increases server security." -authors: ["Ryan Syracuse"] -contributors: ["Ryan Syracuse"] -published: 2021-08-13 -keywords: ["minecraft", "McMyAdmin", "debian", "centos", "ubuntu", "debian 9"] -tags: ["debian, ubuntu, centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[CubeCoders Homepage](https://cubecoders.com/)' - - '[AMP Wiki](https://github.com/CubeCoders/AMP/wiki)' -dedicated_cpu_link: true ---- - -[AMP](https://cubecoders.com/AMP) or **Application Management Panel** is a game server management panel. It provides a graphical overlay to simplify the deployment of game servers, and serves as the next iteration of the popular Minecraft server control panel [McMyAdmin](https://www.mcmyadmin.com/). AMP provides support for third party mods, heavy focus on security, and a sleek web interface for managing your server. This guide covers the installation and configuration of a new AMP server on most Linux distributions. - -While this guide will focus on installation on Debian and Ubuntu, the [same basic process](https://cubecoders.com/AMPInstall) can be applied to other distros with some minor adjustments. - -{{< note >}} -AMP is dependent on the [purchase of a separate license through CubeCoder's website](https://cubecoders.com/AMP#buyAMP). -{{< /note >}} - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. (Optional) Configure a domain to access your management console. For more information on how to do this, see our [DNS Manager Guide](https://techdocs.akamai.com/cloud-computing/docs/dns-manager) - -## Initial Installation of AMP on the Command Line - -AMP provides a startup script that will take care of the basic installation process, and prompt the user for input for their own specific configuration. Enter the following command to initialize the script and begin the setup process: - - bash <(wget -qO- getamp.sh) - -Below is a list of prompts along with an explanation for responses: - -| Prompt | Explanation | Recommended Response | -| ----------- | ----------- | ----------- | -| System Password | This is the Password for the Linux user, not the administrator of the control panel itself. Hit `Enter` to automatically generate a password, or use a secure password of your choice. | Enter a strong password of your choice. | -| Username [admin]: | A username to be used with the control panel. | Enter a username of your choice. | -| Password: | A password for the user of the control panel. | Enter a strong password of your choice. | -| Confirm Password: | A confirmation of the password entered previously for the control panel user. | Re-enter the password entered for the control panel user. | -| Will you be running Minecraft servers on this installation? | Selecting `yes` will install the java dependencies that minecraft servers depend on. | Select `yes` if installing minecraft now or in the future. Select `no` to skip this part of the installation. | -| Will you be running applications that rely on SteamCMD? (Rust, Ark, CSGO, TF2, etc) on this installation? | Selecting `yes` will install the 32-bit libraries that SteamCMD requires. | Selecting `yes` if you plan on installing any applications that may rely on SteamCMD now or in the Future. Select `no` to skip this part of the installation. | -| Would you like to isolate your AMP instances by running them inside Docker containers? | Installing AMP instances inside of a Docker container will help to increase the security of your installation, but may have a small impact on performance | Select `yes` unless you have performance concerns. | -| Would you like AMP to be configured for use with HTTPS? | If you have set up a domain to be used with AMP, then the AMP installation script will be able to detect this and automatically install an SSL/TLS certificate using Let's Encrypt. If you do not have a domain name, then the use of an SSL/TLS certificate is not possible.| Select `yes` if you configured a domain for the server. If not, select `no`, keeping in mind that not using SSL/TLS poses a security risk to data in transit. | - -Once you have responded to all prompts, a message similar to the following will appear, displaying the chosen configuration options: - -{{< output >}} -Installation Summary: - -AMP System user: To be created -Instance Manager: To be installed -HTTPS setup: Yes -Install Java: Yes -Install 32-bit libraries: Yes -Install Docker: Yes -{{< /output >}} - -If the output matches your desired configuration, hit `Enter` for the installation to begin. You should see output on the screen outlining the installation steps as they're completed. - -When the installation script has completed, the following message will appear, prompting you to use your web browser to access the IP address or domain you used for installation over port `8080` to finish the set up process. - -{{< output >}} -[Info] ## Please browse to the instance to complete first-time setup. ## -[Info] Please visit http://example.com:8080 to continue setup. -[Info] -- Do not close this terminal -- -[Info] Waiting for user to complete first-time setup in browser... -{{< /output >}} -{{< note type="alert" >}} -**Do Not** close the terminal or exit the script until the setup process is fully completed in the browser. -{{< /note >}} - -## Completing Setup in the Web Browser - -1. After accessing your domain or IP address over port `8080`, you should see the following login page. - - ![AMP login](amplogin.png) - - Enter the username and password you set up previously and click `Login` to proceed. - -1. Follow the configuration steps as prompted selecting all defaults. Until you get to the section on installing your AMP License key. In order for AMP to work, a [license key must be purchased through CubeCoder's website](https://cubecoders.com/AMP#buyAMP) Enter the license key in the field that appears, and select `next` to proceed. - - ![AMP license](amplicense.png) - -1. Select your desired privacy settings. This includes the option to submit crash reports to AMP and/or to provide in-browser analytics to CubeCoders. When you've made your desired selection, click next to proceed. - - ![AMP privacy](ampprivacy.png) - -Once you've made your choices, the installation will complete and you can begin deploying new game servers. - -## Creating a Game Server - -1. To create a new game server, log into the dashboard and select `Instances` in the sidebar, followed by the `Create Instance` option. - - ![New AMP Instance](ampinstance.png) - -1. In the pop-up menu that appears, select the game you'd like to deploy using the dropdown menu, and select all configuration options as desired. Once all the presets are set, select the `Create Instance` button to create your new server. - - ![AMP application](ampapplication.png) diff --git a/docs/guides/game-servers/launch-a-counter-strike-global-offensive-server-on-ubuntu-18-04/index.md b/docs/guides/game-servers/launch-a-counter-strike-global-offensive-server-on-ubuntu-18-04/index.md deleted file mode 100644 index 53f727a9e2f..00000000000 --- a/docs/guides/game-servers/launch-a-counter-strike-global-offensive-server-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,163 +0,0 @@ ---- -slug: launch-a-counter-strike-global-offensive-server-on-ubuntu-18-04 -title: "Running a Counter Strike Global Offensive Server on Ubuntu 18.04" -title_meta: "How to Run CS:GO Server on Ubuntu 18.04" -description: "This Counter Strike: Global Offensive (CS:GO) server guide explains how to install SteamCMD, download the dedicated server, and launch the game server." -authors: ["Linode"] -contributors: ["Linode"] -published: 2019-01-07 -keywords: ["counter strike", "counter strike global offensive", "csgo", "cs:go", "csgo server", "csgo server hosting", "steam servers", "game servers", "games", "ubuntu", "ubuntu 14.04"] -tags: ["ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[Valve Developer Community - Counter-Strike: Global Offensive Dedicated Servers](https://developer.valvesoftware.com/wiki/Counter-Strike:_Global_Offensive_Dedicated_Servers)' -dedicated_cpu_link: true -relations: - platform: - key: launch-counterstrike-server - keywords: - - distribution: Ubuntu 18.04 ---- - -![Launch a Counter Strike: Global Offensive (CS:GO) server on Ubuntu 18.04](launch-a-counter-strike-global-offensive-1804.png "Launch a Counter Strike: Global Offensive server on Ubuntu 18.04") - -[Counter Strike: Global Offensive](http://store.steampowered.com/app/730) (CS:GO) is a first-person shooter by Valve. Hosting your own server gives you full control over your game and game modes, so you can play the exact flavor of CS:GO you want. This guide contains instructions on how to download the dedicated server and launch the game server. - - - -## Before You Begin - -1. [Create a Linode](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) running Ubuntu 18.04. - -1. Create a [Steam](http://store.steampowered.com) account if you do not have one, and download [Counter Strike: Global Offensive](http://store.steampowered.com/app/730/) to your computer. - -1. A Steam game server login token (GSLT) is required to host a public CS:GO server. Without the token, client connections are restricted to the LAN only. [Register your GSLT](https://steamcommunity.com/dev/managegameservers) on Steam's website. Enter `730` as the App ID when creating your GSLT. Review [Steam's documentation](https://developer.valvesoftware.com/wiki/Counter-Strike:_Global_Offensive_Dedicated_Servers#Registering_Game_Server_Login_Token) for more information about GSLTs. - -1. Complete our guide: [Install SteamCMD for a Steam Game Server](/cloud/guides/install-steamcmd-for-a-steam-game-server). This will get SteamCMD installed and running on your Linode and this guide will pick up where the SteamCMD page leaves off. - - {{< note respectIndent=false >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Prerequisites for Counter-Strike: Global Offensive - -After following the SteamCMD guide, some firewall modifications are needed specifically for CS:GO: - -1. [Stop the SteamCMD process](/cloud/guides/install-steamcmd-for-a-steam-game-server#stop-steamcmd) if it is currently running. - -1. Replace a firewall rule to slightly extend the UDP port range available to the game. This command assumes that you have **only** the iptables rules in place from the SteamCMD guide: - - sudo iptables -R INPUT 5 -p udp -m udp --dport 26900:27030 -j ACCEPT - -1. Reconfigure iptables-persistent to ensure that your new rule persists: - - sudo dpkg-reconfigure iptables-persistent - -## Install Counter Strike: Global Offense - -1. [Run SteamCMD and login to Steam](/cloud/guides/install-steamcmd-for-a-steam-game-server#run-steamcmd) inside a screen session. - -1. From the SteamCMD prompt, install CS:GO to the `steam` user's home directory: - - force_install_dir ./csgo-ds - app_update 740 validate - - This can take some time. If the download looks as if it has frozen, be patient. Once the download is complete, you should see this output: - - Success! App '740' fully installed. - - Steam> - -1. Exit SteamCMD: - - quit - - {{< note respectIndent=false >}} -To update CS:GO, run the above 4 commands again. -{{< /note >}} - -## Configure the Server - -1. Create a file called `server.cfg` using your preferred text editor with the contents of the following snippet. The location you should save this file to depends on how you installed SteamCMD: - - | SteamCMD Installation Method | File Location | - | ------------------- | ------------- | - | Package manager | `~/.steam/steamcmd/csgo-ds/csgo/cfg/server.cfg` | - | Manual installation | `~/Steam/csgo-ds/csgo/cfg/server.cfg` | - - The value for `hostname` will displayed to users that join your server. Replace the values of `sv_password` and `rcon_password` with two different and unique passwords that you don't use elsewhere. - - {{< file "server.cfg" aconf >}} -hostname "The name of your CS:GO server" -sv_password "server_password" -sv_timeout 60 -rcon_password "rcon_password" -mp_autoteambalance 1 -mp_limitteams 1 -writeid -writeip -{{< /file >}} - - `sv_password` is the password users will need to enter to join the server. `rcon_password` is the [RCON](https://developer.valvesoftware.com/wiki/Source_RCON_Protocol) password, which is used to remotely control the game server from within the game. For an extensive list of `server.cfg` options, see [this page](http://csgodev.com/csgodev-server-cfg-for-csgo/). - -1. Create a startup script for CS:GO with the following snippet. Name the file `startcsgo.sh` and save it to your `steam` user's home directory. Set the value of the `YOUR_GSLT` variable at the top to be your game server login token. Set the value of the `CSGO_INSTALL_LOCATION` variable at the top according to the table below. - - {{< file "~/startcsgo.sh" >}} -#!/bin/sh - -YOUR_GSLT= -CSGO_INSTALL_LOCATION= - -cd $CSGO_INSTALL_LOCATION -screen -S "Counter-Strike: Global Offensive Server" ./srcds_run -game csgo -usercon +game_type 0 +game_mode 1 +mapgroup mg_bomb +map de_dust2 +sv_setsteamaccount $YOUR_GSLT -net_port_try 1 -{{< /file >}} - - | SteamCMD Installation Method | CSGO_INSTALL_LOCATION | - | ------------------- | ------------- | - | Package manager | `~/.steam/steamcmd/csgo-ds/` | - | Manual installation | `~/Steam/csgo-ds/` | - - When run, the script will execute a Dust2 server in competitive game mode in a [screen session](/cloud/guides/using-gnu-screen-to-manage-persistent-terminal-sessions). For more startup modes and game options, see Valve's [CS:GO wiki](https://developer.valvesoftware.com/wiki/Counter-Strike:_Global_Offensive_Dedicated_Servers#Starting_the_Server). - -1. Make the script executable: - - chmod +x ~/startcsgo.sh - -## Start the Server - -1. Now that your server is installed and configured, it can be launched by running the `startcsgo.sh` script from your `steam` user's home directory. - - cd ~ && ./startcsgo.sh - -1. Review instructions for [detaching from or stopping SteamCMD](/cloud/guides/install-steamcmd-for-a-steam-game-server#exit-steamcmd) to exit the CS:GO server. - -## Join the Game - -1. Launch Counter-Strike: Global Offensive. - -1. Once launched, go to **Play** and click **Browse Community Servers**. - -1. Click on the **Favorites** tab and then click **Add a Server** at the bottom. - -1. Type in the IP address of your Linode and click **Add this address to favorites**. - -1. You'll see your new Counter-Strike: Global Offensive server. Click **Connect** at the bottom right and start fragging away. - -## Game Settings - -### Game Modes and Types - -You can change the game type and mode options to start different types of servers: - - Mode game_mode game_type - Classic Casual 0 0 - Classic Competitive 0 1 - Arms Race 1 0 - Demolition 1 1 - -These settings are changed in the launch command. - -### RCON - -When logged into the server, you can open the RCON console with the backtick button (`` ` ``), or your mapped key. To log in type `rcon_password` followed by your password. For more information regarding RCON, click [here](/cloud/guides/team-fortress2-on-debian-and-ubuntu#rcon). diff --git a/docs/guides/game-servers/minecraft-with-mcmyadmin-on-debian/index.md b/docs/guides/game-servers/minecraft-with-mcmyadmin-on-debian/index.md deleted file mode 100644 index 390a2e41f81..00000000000 --- a/docs/guides/game-servers/minecraft-with-mcmyadmin-on-debian/index.md +++ /dev/null @@ -1,186 +0,0 @@ ---- -slug: minecraft-with-mcmyadmin-on-debian -title: Installing McMyAdmin for Minecraft on Debian -description: 'In this tutorial, you''ll learn how to install and configure an MCMyAdmin server on a Debian 7 or 8 Linode.' -authors: ["James Stewart"] -contributors: ["James Stewart"] -published: 2015-02-05 -modified: 2019-02-01 -keywords: ["minecraft", "mcmyadmin", "debian", "debian jessie", "debian wheezy", "jessie", "wheezy", "debian 7", "debian 8"] -tags: ["debian"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[McMyAdmin Home Page](https://mcmyadmin.com/)' -aliases: [] -dedicated_cpu_link: true ---- - -![Installing McMyAdmin for Minecraft on Debian](Installing_McMyAdmin_for_Minecraft_on_Debian_smg.png "Installing McMyAdmin for Minecraft on Debian") - -[McMyAdmin](https://mcmyadmin.com/) is one of the most popular Minecraft server control panels available. It boasts compatibility with third party mods, heavy focus on security and a sleek web interface for managing your server. This guide covers the installation and configuration of a new McMyAdmin server on a Linode running Debian 9. Be aware that to actually play on a Minecraft server you must also have the game client from [minecraft.net](https://minecraft.net/). - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. [Mono](http://www.mono-project.com/) is an open source implementation of the .NET framework. CubeCoders Limited, the company behind McMyAdmin, packages its own minimal installation of Mono with some necessary source and configuration files. This must be used instead of the generic Mono packages from Debian's repositories. - - cd /usr/local - apt-get install unzip - wget http://mcmyadmin.com/Downloads/etc.zip - unzip etc.zip; sudo rm etc.zip - -## Configure iptables - -1. Create the files `/tmp/v4` and `/tmp/v6`. Paste the following rulesets into the respective files. - - **IPv4** - - {{< file "/tmp/v4" >}} -*filter - -# Allow all loopback (lo0) traffic and reject traffic -# to localhost that does not originate from lo0. --A INPUT -i lo -j ACCEPT --A INPUT ! -i lo -s 127.0.0.0/8 -j REJECT - -# Allow ping. --A INPUT -p icmp -m state --state NEW --icmp-type 8 -j ACCEPT - -# Allow SSH connections. --A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT - -# Allow connections from other Minecraft players. --A INPUT -p tcp -m state --state NEW --dport 25565 -j ACCEPT - -# Allow web access to McMyAdmin. --A INPUT -p tcp -m state --state NEW --dport 8080 -j ACCEPT - -# Allow inbound traffic from established connections. -# This includes ICMP error returns. --A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT - -# Log what was incoming but denied (optional but useful). --A INPUT -m limit --limit 3/min -j LOG --log-prefix "iptables_INPUT_denied: " --log-level 7 - -# Reject all other inbound. --A INPUT -j REJECT --A FORWARD -j REJECT - -COMMIT -{{< /file >}} - - **IPv6** - - By default, both McMyAdmin and Minecraft operate on IPv4, but unlike a default Minecraft server installation, McMyAdmin does not listen for incoming IPv6 traffic. Since Minecraft can not use both protocols simultaneously, IPv4 is usually chosen over IPv6 because of its much greater availability, thus including players whose ISPs or hardware don't support IPv6. - - If you choose *not* to use IPv6 on your Minecraft server, then it needs only basic IPv6 firewall rules. - - {{< file "/tmp/v6" >}} -*filter - -# Allow all loopback (lo0) traffic and reject traffic -# to localhost that does not originate from lo0. --A INPUT -i lo -j ACCEPT --A INPUT ! -i lo -s ::1/128 -j REJECT - -# Allow ICMP --A INPUT -p icmpv6 -j ACCEPT - -# Allow inbound traffic from established connections. --A INPUT -m state --state ESTABLISHED -j ACCEPT - -# Reject all other inbound. --A INPUT -j REJECT --A FORWARD -j REJECT - -COMMIT -{{< /file >}} - -2. Import the rulesets into immediate use: - - sudo iptables-restore < /tmp/v4 - sudo ip6tables-restore < /tmp/v6 - -3. View the rules that you set: - - sudo iptables -L -nv - sudo ip6tables -L -nv - -4. To apply your iptables rules automatically on boot, see our section on configuring [iptables-persistent](/cloud/guides/control-network-traffic-with-iptables#introduction-to-iptables-persistent). - -## Install Prerequisite Software - -2. Install the Java Runtime Environment, OpenJDK: - - sudo apt-get install openjdk-8-jre - -## Install and Start McMyAdmin - -This section should be completed as your standard user, **not** as root. McMyAdmin will then install to `/home/username`. - -1. Create the installation directory and change location to it. - - sudo mkdir ~/McMyAdmin && cd ~/McMyAdmin - -2. Download the McMyAdmin installer. You will want to double check its [Download](https://www.mcmyadmin.com/#/download) page to be sure you're grabbing the latest version. - - sudo wget http://mcmyadmin.com/Downloads/MCMA2_glibc26_2.zip - -3. Extract the archive and delete the original zip file. - - sudo unzip MCMA2_glibc26_2.zip - sudo rm MCMA2_glibc26_2.zip - -4. Start the initial configuration of McMyAdmin. Replace `PASSWORD` with a strong password which you want for admin access to McMyAdmin's web interface. - - ./MCMA2_Linux_x86_64 -setpass PASSWORD -configonly - - This will return the output: - - The updater will download and install McMyAdmin to the current directory: - /home/your_user/McMyAdmin). - - Continue? [y/n] : - - Answer `y`. The installer will run and return you to the command prompt. If everything is as it should be, the only warning you'll see will be for a missing configuration file. As the output says, that would be normal since McMyAdmin was just started for the first time. - -5. Change into the McMyAdmin installation directory and start the program. - - cd ~/McMyAdmin; ./MCMA2_Linux_x86_64 - - If successful, the last three lines of the output will be: - - Notice : McMyAdmin has started and is ready for use. - Notice : This is the first time McMyAdmin has been started. - Notice : You must complete the first-start wizard via the web interface. - - {{< note respectIndent=false >}} -To exit McMyAdmin and return to the command line, enter `/quit`. -{{< /note >}} - -## Managing your Minecraft Server - -1. Browse to the McMyAdmin web interface by visiting `http://YourLinodeIP:8080`. - -2. Log in with the username `admin` and the password that you provided in the installation step. - - ![McMyAdmin Login Page](mcmyadmin-login-page.png) - -3. Once the initial configuration steps are completed, select your settings and then switch to the status page. - - ![McMyAdmin Configuration Page](mcmyadmin-config-page.png) - -4. Select *Start Server* and accept the Minecraft Server EULA (End User Licensing Agreement). - - {{< note respectIndent=false >}} -If you are not prompted to accept the EULA in McMyAdmin, you can find the EULA at `~/McMyAdmin/Minecraft/eula.txt`. Change the value of `eula=false` to `eula=true`. -{{< /note >}} - - ![McMyAdmin Status Page](mymyadmin-status-page.png) - - ![McMyAdmin Server Started](mcmyadmin-server-running.png) - -Congratulations, you now have McMyAdmin running on your Minecraft server! diff --git a/docs/guides/game-servers/multicraft-on-ubuntu/index.md b/docs/guides/game-servers/multicraft-on-ubuntu/index.md deleted file mode 100644 index 67f40fcf0db..00000000000 --- a/docs/guides/game-servers/multicraft-on-ubuntu/index.md +++ /dev/null @@ -1,177 +0,0 @@ ---- -slug: multicraft-on-ubuntu -title: "Installing Multicraft on Ubuntu" -description: "This guide shows how to install and configure MultiCraft, a control panel for single or multiple Minecraft servers, on a Linode running Ubuntu 20.04 LTS." -authors: ["Alex Fornuto"] -contributors: ["Alex Fornuto"] -published: 2015-02-04 -modified: 2021-12-30 -keywords: ["minecraft", "ubuntu", "multicraft"] -tags: ["ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Multicraft Documentation](http://www.multicraft.org/site/userguide?view=index)' - - '[Minecraft.net](https://minecraft.net/)' -image: Installing_Multicraft_on_Ubuntu_smg.jpg -aliases: [] -dedicated_cpu_link: true -relations: - platform: - key: how-to-install-multicraft - keywords: - - distribution: Ubuntu ---- - -[Multicraft](http://www.multicraft.org/) is a control panel for single or multiple Minecraft servers, with free and paid versions available. This guide provides information to install Multicraft on a Linode running Ubuntu 20.04 LTS. - -{{< note >}} -The steps required in this guide require root privileges. Be sure to run the steps as `root` or with the **sudo** prefix. For more information on privileges see our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Prerequisites - -Multicraft for Linux depends on several software packages in order to run. - -1. Update your system: - - sudo apt update && sudo apt upgrade - -1. Install LAMP server: - - sudo apt install tasksel - sudo tasksel install lamp-server - -1. Install PHP, SQLite, Java, and related packages: - - sudo apt install -y zip php-zip php php-sqlite3 php-gd default-jre - - -1. In configuration file of Apache, under the `` section, change the `AllowOverride` value to `all`. - - {{< file "/etc/apache2/apache2.conf" apache >}} - - Options Indexes FollowSymLinks - AllowOverride All - Require all granted - -{{< /file >}} - -1. Reload the Apache configuration: - - sudo service apache2 reload - -## Installing Multicraft - -1. Download the Multicraft installer: - - wget http://www.multicraft.org/download/linux64 -O multicraft.tar.gz - -1. Expand the installer: - - tar -xzf multicraft*.tar.gz - -1. Move to the `multicraft` directory: - - cd multicraft/ - -1. Execute the `setup.sh` script: - - ./setup.sh - -1. An interactive script that prompts you to configure several options runs. If you've purchased a license for Multicraft, enter it when prompted. If you're unsure of which options to choose, you can press `return` to select the default option, with a couple of exceptions: - - * Unless you're configuring a custom Apache virtual host for Multicraft, specify the location for the PHP frontend to `/var/www/html/multicraft`: - - Location of the PHP frontend: [/var/www/multicraft] /var/www/html/multicraft - - * Because of the insecure nature of FTP, Linode recommends that you **not** enable the builtin FTP server when prompted: - - Enable builtin FTP server? [y]/n n - - After the script finishes, you can begin configuring the Multicraft install. - -1. Make a copy the file configuration file and access to the `www-data` user. - - cp /var/www/html/multicraft/protected/config/config.php.dist /var/www/html/multicraft/protected/config/config.php then - chown www-data:www-data /var/www/html/multicraft/protected/config/config.php - -1. Enable the ports to access the server: - - sudo apt install ufw - ufw allow 80 - ufw allow 443 - ufw allow 22 - ufw enable - - After the ports are enabled, restart the server. - -## Configuring the Control Panel - -1. In your local web browser, navigate to `http://192.0.2.0/multicraft`, replacing `192.0.2.0` with your Linode's IP address or domain name. Click `Start Installation`: - - ![Multicraft Installer.](multicraft-init-1804.png) - -1. Multicraft checks the requirements. If you completed the steps above without issue, the following page appears: - - ![Multicraft Requirements Check.](multicraft-reqs-2004.png) - - Click `Continue`. - -1. Multicraft checks for the `config.php` file and if it is writeable. If successful, click `Continue`: - - ![Multicraft Configuration File Transfer.](multicraft-config-2004.png) - -1. On the next page, click `Initialize Database`. - - ![Multicraft Database Initialization.](multicraft-db-initialize-1804.png) - -1. Afterward the database is initialized, click `Continue`: - - ![Multicraft Database Creation.](multicraft-db-1804.png) - -1. The next page attempts to connect to the panel database. You should see the message `Connection successful`. You can now click on the `Login` button and sign in with the default username and password as **admin**. - - ![Multicraft Panel Database Connection.](multicraft-panel-1804.png) - -1. After logging in you are directed back to the previous page, where you can now click `Continue`. The next page allows you to configure the basic settings. When done, click `Save`. - -1. On the daemon configuration page, a start command to initiate the Multicraft Daemon appears. Copy the command into your terminal: - - /home/minecraft/multicraft/bin/multicraft -v start - Multicraft 2.4.1 - Minecraft Server Manager Daemon - Loading configuration from /home/minecraft/multicraft/multicraft.conf - Starting daemon - !! You are using the default daemon password. - !! Please change the "password" setting in the "multicraft.conf" as well as the "daemon_password" setting in the "protected/config/config.php" of the panel - -1. Back in the browser, click `Refresh`. You should see the daemon in the detected daemons list. Click `Continue`: - - ![Multicraft Daemon Configuration.](multicraft-daemon-1804.png) - -1. Your configuration of the Multicraft control panel is now complete. As per the instructions on the page, delete the `install.php` file from your terminal: - - rm /var/www/html/multicraft/install.php - -## Install Minecraft - -1. Navigate to the directory where Multicraft stores `jar` files. If you used the default options, it is `/home/minecraft/multicraft/jar/`: - - cd /home/minecraft/multicraft/jar/ - -1. Download the latest version of the Minecraft server from the Minecraft [Download](https://minecraft.net/download) page: - - wget https://launcher.mojang.com/v1/objects/0a269b5f2c5b93b1712d0f5dc43b6182b9ab254e/server.jar - -Version 1.17 (minecraft_server.1.17.jar) is downloaded. - -1. In your local web browser, navigate to `http://192.0.2.0/multicraft/`, replacing `192.0.2.0` with your Linode's IP address or domain name. Click `Servers`. - -1. At this time you must accept the Multicraft EULA. A pop-up window appears. By clicking Close you are indicating your agreement to the EULA here: `http://www.multicraft.org/eula.txt`. Click Close. - - ![Multicraft EULA.](multicraft-eula-popup.png) - -1. Click `Create Server`. Fill in the options as you see fit, but be sure to add `server.jar` (or the version that you downloaded) in the `JAR File` field: - - ![Multicraft Server Settings.](multicraft-server-settings-1804.png) - - You can now successfully start and manage your Minecraft server through Multicraft! For more information, see the [Connect to your Minecraft Server](/cloud/guides/how-to-set-up-minecraft-server-on-ubuntu-or-debian#connect-to-your-minecraft-server) section on the [How to Set Up a Minecraft Server on Ubuntu or Debian](/cloud/guides/how-to-set-up-minecraft-server-on-ubuntu-or-debian) guide. diff --git a/docs/guides/kubernetes/deploy-minio-on-kubernetes-using-kubespray-and-ansible/index.md b/docs/guides/kubernetes/deploy-minio-on-kubernetes-using-kubespray-and-ansible/index.md deleted file mode 100644 index c399ddc95e0..00000000000 --- a/docs/guides/kubernetes/deploy-minio-on-kubernetes-using-kubespray-and-ansible/index.md +++ /dev/null @@ -1,416 +0,0 @@ ---- -slug: deploy-minio-on-kubernetes-using-kubespray-and-ansible -title: 'Deploy Minio on Kubernetes using Kubespray and Ansible' -description: 'Learn how to use a combination of Kubespray and Ansible to provision a cluster and deploy Minio as a private cloud storage.' -authors: ["Sam Foo"] -contributors: ["Sam Foo"] -published: 2018-02-23 -keywords: ['ansible', 'kubernetes', 'cluster', 's3', 'aws'] -tags: ["python","kubernetes","automation"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -concentrations: ["Kubernetes"] -external_resources: -- '[Kubernetes](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/)' -- '[Minio](https://www.minio.io/)' -- '[Kubespray](https://github.com/kubernetes-incubator/kubespray)' ---- - -![Header image](Deploy_Minio.jpg "Deploy Minio on Kubernetes using Kubespray and Ansible") - -## What is Minio? - -Minio is an open source, Amazon S3-compatible object store that can be hosted on a Linode. Deployment on a Kubernetes cluster is supported in both standalone and distributed modes. This guide uses [Kubespray](https://github.com/kubernetes-incubator/kubespray) to deploy a Kubernetes cluster on three servers running Ubuntu 16.04. Kubespray comes packaged with Ansible playbooks that simplify setup on the cluster. Minio is then installed in standalone mode on the cluster to demonstrate how to create a service. - -## Before You Begin - -1. For demonstration purposes, this guide installs `etcd` and the Kubernetes master on the same node. High availability clusters will require a different configuration, which is beyond the scope of this guide. - -1. Each Linode to be used in the cluster should have a user with sudo privileges. - -1. A cluster can be simulated locally using [Minikube](https://github.com/kubernetes/minikube) to get comfortable with Kubernetes clusters. - -1. The IP addresses of each node in the cluster and their roles will be represented as `kubernetes-master-ip`, `etcd-ip`, and `slave-ip` - -{{< note >}} -If you do not want to install Ansible and other software locally, consider using another Linode as a jumpbox that will be used to connect with the master node. -{{< /note >}} - -## Install Ansible - -1. Update if needed. - - ```command - sudo apt-get update - sudo apt-get install software-properties-common - ``` - -1. Add the Ansible PPA; press enter when prompted. - - ```command - sudo apt-add-repository ppa:ansible/ansible - ``` - - ```output - Ansible is a simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems. - - http://ansible.com/ - More info: https://launchpad.net/~ansible/+archive/ubuntu/ansible - Press [ENTER] to continue or ctrl-c to cancel adding it - - gpg: keyring `/tmp/tmp81pkp_0b/secring.gpg' created - gpg: keyring `/tmp/tmp81pkp_0b/pubring.gpg' created - gpg: requesting key 7BB9C367 from hkp server keyserver.ubuntu.com - gpg: /tmp/tmp81pkp_0b/trustdb.gpg: trustdb created - gpg: key 7BB9C367: public key "Launchpad PPA for Ansible, Inc." imported - gpg: Total number processed: 1 - gpg: imported: 1 (RSA: 1) - OK - ``` - -1. Update again then install Ansible. - - ```command - sudo apt-get update - sudo apt-get install ansible - ``` - -### Additional Installation - -Kubespray exists as a Git repository and requires `python-netaddr` for network address manipulation. - -1. Install Git: - - ```command - sudo apt install git - ``` - -1. Install `python-netaddr`: - - ```command - sudo apt install python-netaddr - ``` - -## Modify Kubespray Configurations - -Kubespray comes with several configuration options not shown in this guide. Refer to the [documentation](https://kubespray.io/documents/) for more information on topics such as networking with Flannel, Helm installation, and large scale deployments. - -1. Clone the Kubespray repository from Github then navigate into the repository. - - ```command - git clone https://github.com/kubernetes-incubator/kubespray.git - cd kubespray - ``` - -1. Check out a tag for the desired version of Kubespray. This guide is written for version 2.4.0. - - ```command - git checkout -b tag/v.2.4.0 - ``` - -1. Modify `~/kubespray/ansible.cfg` to run Ansible playbooks on hosts as a given user. Replace `username` with your Unix account username in `remote_user=username` under `[defaults]`. - - ```file {title="~/kubespray/ansible.cfg" lang=cfg} - [ssh_connection] - pipelining=True - ssh_args = -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null - #control_path = ~/.ssh/ansible-%%r@%%h:%%p - [defaults] - host_key_checking=False - gathering = smart - fact_caching = jsonfile - fact_caching_connection = /tmp - stdout_callback = skippy - library = ./library - callback_whitelist = profile_tasks - roles_path = roles:$VIRTUAL_ENV/usr/local/share/kubespray/roles:$VIRTUAL_ENV/usr/local/share/ansible/roles:/usr/share/kubespray/roles - deprecation_warnings=False - remote_user=username - ``` - -1. Copy the example inventory directory and rename it: - - ```command - cp -r inventory/sample inventory/minio - ``` - -1. Use Kubespray's inventory generator to build an inventory of hosts for Ansible. Declare the list of IP addresses for each Linode. - - ```command - declare -a IPS=(kubernetes-master-ip etcd-ip slave-ip) - CONFIG_FILE=inventory/minio/hosts.ini python3 contrib/inventory_builder/inventory.py ${IPS[@]} - ``` - - {{< note >}} - Do not use hostnames when declaring `$IPS`. Only IP addresses are supported by the inventory generator at this time. - {{< /note >}} - -1. Example configuration for the cluster in this guide. - - ```file {title="~/kubespray/inventory/minio/hosts.ini" lang=ini} - [all] - node1 ansible_host=kubernetes-master-ip ip=kubernetes-master-ip - node2 ansible_host=etcd-ip ip=etcd-ip - node3 ansible_host=slave-ip ip=slave-ip - - [kube-master] - node1 - - [kube-node] - node2 - node3 - - [etcd] - node1 - - [k8s-cluster:children] - kube-node - kube-master - - [calico-rr] - - [vault] - node1 - node2 - node3 - ``` - -1. Uncomment the line `docker_dns_servers_strict: false` in `~/kubernetes/inventory/minio/group_vars/all.yml` - -## Prepare Hosts for Ansible -Before Ansible can properly run Kubespray's playbooks, the hosts must have a passwordless sudo user enabled, and swap disabled for Kubernetes. Make sure the specified user exists on each Linode prior to starting these steps. This section shows how to copy SSH keys to each Linode and modify the sudoers file over SSH. - -1. Create a private key if you **do not** have one: - - ```command - ssh-keygen -b 4096 - ``` - -1. Copy your SSH key to each IP listed in the inventory using the `$IPS` variable declared earlier and replace `username` with the username for each of the hosts. - - ```command - for IP in ${IPS[@]}; do ssh-copy-id username@$IP; done - ``` - -### Create Passwordless Sudo on Nodes - -Below is a loop that adds the line `username ALL=(ALL:ALL) NOPASSWD: ALL` to the last line of the sudoers file. You will be prompted for the password for each server. - -```command -for IP in ${IPS[@]}; do ssh -t username@$IP "echo 'username ALL=(ALL:ALL) NOPASSWD: ALL' | sudo EDITOR='tee -a' visudo"; done -``` - -### Disable swap - -Add this snippet below at the end of `~/kubespray/roles/bootstrap-os/tasks/main.yml` to disable swap using Ansible. - -```file {title="~/kubespray/roles/bootstrap-os/tasks/main.yml" lang=yaml} -- name: Remove swapfile from /etc/fstab - mount: - name: swap - fstype: swap - state: absent - -- name: Disable swap - command: swapoff -a -``` - -## Run Ansible Playbook - -Before running the Ansible playbook, make sure firewalls are turned off to avoid unexpected errors. - -Run the `cluster.yml` Ansible playbook. If your private key is named differently or located elsewhere, add `--private-key=/path/to/id_rsa` to the end. - -```command -ansible-playbook -i inventory/minio/hosts.ini cluster.yml -b -v -``` - -{{< note >}} -This could take up to 20 minutes. -{{< /note >}} - -### Add or Remove Nodes - -1. Navigate into `~/kubespray/inventory/minio/hosts.ini` and add the IP address of the new node. - -1. Run ssh-copy-id to copy your SSH key to the new node: - - ```command - ssh-copy-id username@new-node-ip - ``` - -1. Run the `scale.yml` Ansible playbook: - - ```command - ansible-playbook -i inventory/minio/hosts.ini scale.yml -b -v - ``` - -1. SSH into the Kubernetes master node to list all the available nodes: - - ```command - kubectl get nodes - ``` - -1. To remove a node, simply turn off the server and clean up on the master node with: - - ```command - kubectl delete node - ``` - -## Minio on Kubernetes - -The commands in this section should be executed from the `kubernetes-master` Linode. - -### Create a Persistent Volume - -Persistent Volumes(PV) are an abstraction in Kubernetes that represents a unit of storage provisioned in the cluster. A `PersistentVolumeClaim`(PVC) will allow a Pod to consume the storage set aside by a PV. This section creates a PV of 15Gi ([gibibytes](https://en.wikipedia.org/wiki/Binary_prefix)) then allow Minio to claim 10Gi of space. - -1. On the Kubernetes master node, create a file called `minio-volume.yaml` with the following YAML below. Replace `username` on the `hostPath` with the appropriate path. - - ```file {title="minio-volume.yaml" lang=yaml} - kind: PersistentVolume - apiVersion: v1 - metadata: - name: minio-pv-volume - labels: - type: local - spec: - storageClassName: manual - capacity: - storage: 15Gi - accessModes: - - ReadWriteOnce - hostPath: - path: "/home/username" - ``` - -1. Create the PV: - - ```command - kubectl create -f minio-volume.yaml - ``` - -1. Create a PVC with `minio-pvc.yaml`: - - ```file {title="minio-pvc.yaml" lang=yaml} - apiVersion: v1 - kind: PersistentVolumeClaim - metadata: - name: minio-pv-claim - labels: - app: minio-storage-claim - spec: - storageClassName: manual - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 10Gi - ``` - -1. Create the PVC: - - ```command - kubectl create -f minio-pvc.yaml - ``` - -### Create a Deployment - -1. Create a Deployment configuration in `minio-deployment.yaml` and substitute `username` on the last line. The access and secret key are in the YAML file. - - ```file {title="minio-deployment.yaml" lang=yaml} - apiVersion: apps/v1 # for k8s versions before 1.9.0 use apps/v1beta2 and before 1.8.0 use extensions/v1beta1 - kind: Deployment - metadata: - # This name uniquely identifies the Deployment - name: minio-deployment - spec: - selector: - matchLabels: - app: minio - strategy: - type: Recreate - template: - metadata: - labels: - # Label is used as selector in the service. - app: minio - spec: - # Refer to the PVC created earlier - volumes: - - name: storage - persistentVolumeClaim: - # Name of the PVC created earlier - claimName: minio-pv-claim - containers: - - name: minio - # Pulls the default Minio image from Docker Hub - image: minio/minio:latest - args: - - server - - /storage - env: - # Minio access key and secret key - - name: MINIO_ACCESS_KEY - value: "minio" - - name: MINIO_SECRET_KEY - value: "minio123" - ports: - - containerPort: 9000 - hostPort: 9000 - # Mount the volume into the pod - volumeMounts: - - name: storage # must match the volume name, above - mountPath: "/home/username" - ``` - -1. Create the Deployment. - - ```command - kubectl create -f minio-deployment.yaml - ``` - -### Create a Service - -1. Create a file for the service called `minio-service.yaml` - - ```file {title="minio-service.yaml" lang=yaml} - apiVersion: v1 - kind: Service - metadata: - name: minio-service - spec: - type: LoadBalancer - ports: - - port: 9000 - targetPort: 9000 - protocol: TCP - selector: - app: minio - ``` - -1. Deploy the Minio service: - - ```command - kubectl create -f minio-service.yaml - ``` - -1. See a list of running services. Under the column `PORT(S)`, you can see that the Minio service is running internally on port 9000, with 30593 exposed externally by the LoadBalancer. - - ```command - kubectl get services - ``` - - ```output - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.233.0.1 443/TCP 1d - minio-service LoadBalancer 10.233.28.163 9000:30593/TCP 20m - ``` - -1. In a browser, navigate to the public IP address of any of the Linodes in the cluster, at the exposed port (30593 in the example above): - - ![Minio Login Screen](minio-login-screen.png) - -1. Minio has similar functionality to Amazon S3: file uploads, creating buckets, and storing other data. - - ![Minio Browser](minio-browser.png) \ No newline at end of file diff --git a/docs/guides/kubernetes/how-to-deploy-jenkins-x-in-linode-kubernetes-engine/index.md b/docs/guides/kubernetes/how-to-deploy-jenkins-x-in-linode-kubernetes-engine/index.md deleted file mode 100644 index 7ae02dbffba..00000000000 --- a/docs/guides/kubernetes/how-to-deploy-jenkins-x-in-linode-kubernetes-engine/index.md +++ /dev/null @@ -1,341 +0,0 @@ ---- -slug: how-to-deploy-jenkins-x-in-linode-kubernetes-engine -title: "Deploying Jenkins X in Linode Kubernetes Engine" -title_meta: "How to Deploy Jenkins X in Linode Kubernetes Engine" -description: 'Learn how to install CI/CD tool Jenkins X on Linode Kubernetes Engine in this step-by-step guide.' -authors: ["Daniele Polencic"] -contributors: ["Daniele Polencic"] -published: 2020-07-29 -keywords: ['kubernetes','pipelines','ci/cd','kubernetes','jenkins x','jenkins'] -tags: ["docker","kubernetes","container","automation","linode platform"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: 'learnk8s_jenkinsX.png' -external_resources: -- '[How to run Jenkins X Boot](https://jenkins-x.io/docs/install-setup/boot/how-it-works/)' -- '[Creating and importing projects in Jenkins X](https://jenkins-x.io/docs/create-project/creating/)' -aliases: [] -deprecated: true ---- - -## What is Jenkins X? - -[Jenkins X](https://jenkins-x.io) is CI/CD tool that automates the management of development environments and the promotion of new versions of applications between environments. This guide illustrates how to install Jenkins X version 2 on Linode Kubernetes Engine. - -{{< note >}} -Jenkins X version 2 is no longer maintained. For more information about using Jenkins X version 3, see the [official Jenkins X documentation](https://jenkins-x.io/v3/admin/). -{{< /note >}} - -## Before You Begin - -1. You will need a personal GitHub account. If you don't have one, [you can register here](https://github.com/join). -1. You need a GitHub account for your bot. This bot automates actions like triaging issues, merging Pull Requests, etc. [You can register your bot account here](https://github.com/join). -1. You will need an account on DockerHub. If you don't have one, [you can register here](https://hub.docker.com/signup). - -## Create an LKE Cluster - -Follow the instructions in [Deploying and Managing a Cluster with Linode Kubernetes Engine Tutorial](https://techdocs.akamai.com/cloud-computing/docs/linode-kubernetes-engine) to create and connect to an LKE cluster. - -We recommend using three 16GB Linodes to start with. - -![Creating a Kubernetes cluster on Linode](create-cluster.png) - -You can verify that the installation is successful with: - -```command -kubectl get nodes -``` - -The output should similar to: - -```output -NAME STATUS ROLES AGE VERSION -lke7189-9006-5f05145fc9a3 Ready 8h v1.17.3 -lke7189-9006-5f051460a1e2 Ready 8h v1.17.3 -lke7189-9006-5f0514617a87 Ready 8h v1.17.3 -``` - -## Install jx - -To install Jenkins X, you need the `jx` command-line tool. You can find the instruction on [how to install `jx` on the official documentation.](https://jenkins-x.io/docs/install-setup/install-binary/) - -You can verify that the installation is successful with: - -```command -jx version -``` - -The output should be similar to: - -```output -Version 2.1.80 -Commit b2bd447 -Build date 2020-06-23T15:34:17Z -Go version 1.13.8 -Git tree state clean -``` - -## Bootstrap Jenkins X - -{{< note >}} -If you used the `export` command with the KUBECONFIG variable so that you can use kubectl to connect to your cluster, you will need to perform the following steps in the same terminal window where you executed that command in order for `jx` to also be able to connect to your cluster. -{{< /note >}} - -1. Create a new folder, navigate to the new folder, and execute the `jx boot` command to start a new project: - - ```command - mkdir jenkins-x - cd jenkins-x - jx boot - ``` - - The command prompts the following message: - - ```output - Creating boot config with defaults, as not in an existing boot directory with a git repository. - No Jenkins X pipeline file jenkins-x.yml or no jx boot requirements file jx-requirements.yml found. You are not running this command from inside a Jenkins X Boot git clone - To continue we will clone https://github.com/jenkins-x/jenkins-x-boot-config.git @ master to jenkins-x-boot-config - ? Do you want to clone the Jenkins X Boot Git repository? [? for help] (Y/n) - ``` - - _Yes_ is the default answer, so you can press enter and continue to the next step. - -1. In the next step, the CLI clones the Jenkins X configuration repo and validates your configuration. - - The command-line tool expects the cluster name to be in the format `__` — which is the default if you want to use Google Kubernetes Engine. Since Linode Kubernetes Engine doesn't follow the same naming pattern, the validation fails: - - ```output - Cloning https://github.com/jenkins-x/jenkins-x-boot-config.git @ master to jenkins-x-boot-config - Attempting to resolve version for boot config https://github.com/jenkins-x/jenkins-x-boot-config.git from https://github.com/jenkins-x/jenkins-x-versions.git - Booting Jenkins X - - STEP: validate-git command: /bin/sh -c jx step git validate in dir: /tmp/jenkinsx/jenkins-x-boot-config/env - - STEP: verify-preinstall command: /bin/sh -c jx step verify preinstall --provider-values-dir="kubeProviders" in dir: /tmp/jenkinsx/jenkins-x-boot-config - - error: : unable to parse lke7389 as <project id>_<zone>_<cluster name> - error: failed to interpret pipeline file /tmp/jenkinsx/jenkins-x-boot-config/jenkins-x.yml: failed to run '/bin/sh -c jx step verify preinstall --provider-values-dir="kubeProviders"' command in directory '/tmp/jenkinsx/jenkins-x-boot-config', output: '' - ``` - - But don't worry, this error can be fixed. If you inspect the current folder, you should notice that `jx` created a `jenkins-x-boot-config` folder. In that folder, there's a `jx-requirements.yml` file with the details of your cluster. - - Open that with your favourite editor and identify the line that starts with: `provider: gke`. You should change that to `provider: kubernetes`. - - ```file {title="jx-requirements.yml" lang=yaml} - autoUpdate: - enabled: false - schedule: "" - bootConfigURL: https://github.com/jenkins-x/jenkins-x-boot-config.git - cluster: - gitKind: github - gitName: github - gitServer: https://github.com - namespace: jx - provider: gke # <- you should change this line to "kubernetes"! - ``` - -1. It's time to run the `jx boot` command again, but this time from within the `jenkins-x-boot-config` folder: - - ```command - cd jenkins-x-boot-config - jx boot - ``` - - This time, the command stops at a different step, which has you confirm you'd like to run `jx` on a service that is not GKE or EKS: - - ```output - Attempting to resolve version for boot config https://github.com/jenkins-x/jenkins-x-boot-config from https://github.com/jenkins-x/jenkins-x-versions.git - Booting Jenkins X - - STEP: validate-git command: /bin/sh -c jx step git validate in dir: /tmp/jenkinsx/jenkins-x-boot-config/env - - STEP: verify-preinstall command: /bin/sh -c jx step verify preinstall --provider-values-dir="kubeProviders" in dir: /tmp/jenkinsx/jenkins-x-boot-config - - jx boot has only been validated on GKE and EKS, we'd love feedback and contributions for other Kubernetes providers - ? Continue execution anyway? (Y/n) - ``` - - _Yes_ is the default answer, so press enter and continue executing. - -1. The next question is the cluster name: - - ```output - ? Cluster name - ``` - - Enter `jenkins-x-lke`. - -1. Next, make yourself the Git Owner. - - ```output - ? Git Owner name for environment repositories - ``` - - Type in your GitHub username, and press enter. - -1. In the next step, name the users you wish to associate as an approver. This should include your username and the names of any other members of your team which you wish to grant this ability, and the input should be a comma separated list. - - ```output - Environment repos will be private, if you want to create public environment repos, please set environmentGitPublic to true in jx-requirements.yml - ? Comma-separated git provider usernames of approvers for development environment repository - ``` - -1. Pay attention to the next question, which describes that the webhooks will be called over HTTP, not HTTPS: - - ```output - Locking version stream https://github.com/jenkins-x/jenkins-x-versions.git to release v1.0.529. Jenkins X will use this release rather than master to resolve all versions from now on. - writing the following to the OWNERS file for the development environment repository: - approvers: - - <your_GitHub_username> - reviewers: - - <your_GitHub_username> - WARNING: TLS is not enabled so your webhooks will be called using HTTP. This means your webhook secret will be sent to your cluster in the clear. See https://jenkins-x.io/docs/getting-started/setup/boot/#ingress for more information - ? Do you wish to continue? [? for help] (y/N) - ``` - - Notice how _No_ is the default answer. However, you should answer **Yes** and move on to the next question. - -1. In the next steps, `jx` configures several components in your cluster. It might take a while before you're requested to interact with the terminal. When you do, the next questions are about the Jenkins X admin: - - ```output - ? Jenkins X Admin Username (admin) - ? Jenkins X Admin Password - ``` - - You should choose a name and a password for the admin user. - -1. Next, you enter in the info about your Jenkins X Bot. - - The bot will help you triage issues, Pull Requests and execute chores. In this part, you should enter the details for your bot (the GitHub account that you created in advance): - - ```output - ? Pipeline bot Git username - ? Pipeline bot Git email address - ? Pipeline bot Git token - ``` - - You can create a token for your bot by [visiting this URL](https://github.com/settings/tokens/new?scopes=repo,read:user,read:org,user:email,write:repo_hook,delete_repo). - - {{< note >}} - Please note that you should create the token with the Bot's account and not yours. - {{< /note >}} - - ![Creating an API token on GitHub](github-token.png) - -1. You can use the same token for the next question (and just press enter): - - ```output - ? HMAC token, used to validate incoming webhooks. Press enter to use the generated token - ``` - -1. The next question is crucial, as it concerns how Jenkins X will interact with the docker registry: - - ```output - ? Do you want to configure non default Docker Registry? (y/N) - ``` - - Linode does not offer a container registry at the moment, so you will use Docker Hub as your container registry. You should answer _Yes_ which is **NOT** the default option. Answer the following questions accordingly: - - ```output - ? Do you want to configure non default Docker Registry? Yes - ? Docker Registry Url <accept-the-default-value> - ? Docker Registry username <enter-your-dockerhub-username> - ? Docker Registry password <enter-your-dockerhub-password> - ? Docker Registry email <enter-your-dockerhub-email> - ``` - -The installation should complete with the following output: - -```output -Verifying the git config -Verifying username <bot-name> at git server github at https://github.com -Found 1 organisation in git server https://github.com: learnk8s -Validated pipeline user <bot-name> on git server https://github.com -Git tokens seem to be setup correctly -Installation is currently looking: GOOD -Using namespace 'jx' from context named 'lke7411-ctx' on server 'https://c50d6328-b182-4fe9-9746-7f6e2b2e1b4d.ap-south-1.linodelke.net:443'. -``` - -Jenkins X is installed! - -If you inspect your GitHub profile, you might have noticed that there are three new private repositories: - -- `environment-jenkins-x-lke-dev` -- `environment-jenkins-x-lke-staging` -- `environment-jenkins-x-lke-prod` - -The repositories hold the configuration for each environment. - -## Your First Project - -In this part, you will set up a CI/CD pipeline for a Java (Spring Boot) application. The same steps are valid for any other language or framework. - -1. You should fork the [Jenkins X Demo repository from LearnK8s](https://github.com/learnk8s/jenkins-x-demo). - -1. Next, clone the repository locally and change your current directory, being sure to add your GitHub username to the cloned repository's address: - - ```command - git clone git@github.com:your-github-username/jenkins-x-demo - cd jenkins-x-demo - ``` - -1. You can import the project in Jenkins X with the following command: - - ```command - jx import - ``` - - You will see a similar output to this: - - ```output - PipelineActivity for <your-gh-username>-jenkins-x-demo-master-1 - upserted PipelineResource meta-<your-gh-username>-jenkins-x-dlncr for the git repository https://github.com/<your-gh-username>/jenkins-x-demo.git - upserted Task meta-<your-gh-username>-jenkins-x-dlncr-meta-pipeline-1 - upserted Pipeline meta-<your-gh-username>-jenkins-x-dlncr-1 - created PipelineRun meta-<your-gh-username>-jenkins-x-dlncr-1 - created PipelineStructure meta-<your-gh-username>-jenkins-x-dlncr-1 - - Watch pipeline activity via: jx get activity -f jenkins-x-demo -w - Browse the pipeline log via: jx get build logs <your-gh-username>/jenkins-x-demo/master - You can list the pipelines via: jx get pipelines - When the pipeline is complete: jx get applications - - For more help on available commands see: https://jenkins-x.io/developing/browsing/ - ``` - -1. You can now run the following command to follow the pipeline output: - - ```command - jx get build logs /jenkins-x-demo/master - ``` - - The pipeline will take around three or four minutes to complete, as it has to: - - - Compile the project. - - Create tags in the repository. - - Build and push the container image to Docker Hub. - - Package the Helm chart for the application. - -1. In the last step of the pipeline, Jenkins X raises a Pull Request to the repository that holds the configuration of your staging environment (created by default during installation). - - At the end of the pipeline you will see the following output: - - ```output - Created Pull Request: https://github.com//environment-jenkins-x-lke-staging/pull/1 - ``` - - The Pull Request is an automatic promotion to the staging environment. If you merge the Pull Request, the application is automatically promoted to the staging environment. - - You can merge the Pull Request to see the pipeline applying the changes in the staging environment. - - If the promotion is successful, you can inspect the app with: - - ```command - jx get applications - ``` - - ```output - APPLICATION STAGING PODS URL - jenkins-x-demo 0.0.10 http://jenkins-x-staging.178.79.175.247.nip.io - ``` - -It might take some time for your application to start. However, when it's started, you can visit the app at the URL provided and see it running. \ No newline at end of file diff --git a/docs/guides/kubernetes/how-to-install-rooknfs-on-lke/index.md b/docs/guides/kubernetes/how-to-install-rooknfs-on-lke/index.md deleted file mode 100644 index 54b46eb9ec8..00000000000 --- a/docs/guides/kubernetes/how-to-install-rooknfs-on-lke/index.md +++ /dev/null @@ -1,285 +0,0 @@ ---- -slug: how-to-install-rooknfs-on-lke -title: "Setting Up Rook NFS for Persistent Storage on LKE" -title_meta: "How to Set Up Rook NFS for Persistent Storage on LKE" -description: 'How to install Rook NFS on LKE.' -authors: ["Todd Becker"] -contributors: ["Todd Becker"] -published: 2021-08-13 -keywords: ['rooknfs','kubernetes', 'lke', 'linode kubernetes engine'] -tags: ["docker","kubernetes","container","nginx","linode platform"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -deprecated: true ---- - -Rook NFS allows remote hosts to mount filesystems over a network and interact with those filesystems as though they are mounted locally. When used with LKE, Rook can mount a Linode Block Storage PVC which uses `ReadWriteOnce` permissions. The volume can then be leveraged as NFS and exported as a storage class that uses `ReadWriteMany` permissions. This allows Linode's Block Storage to store persistent data for LKE clusters. - -## Before you Begin - -- This guide assumes that you already have an LKE cluster up and running. If that is not the case, please follow the instructions in our [LKE Tutorial](https://techdocs.akamai.com/cloud-computing/docs/linode-kubernetes-engine). -- This guide relies on git wherever `kubectl` is installed. While git is installed on many Linux distributions, others may require manual installation. Git can be installed on most distributions by following our [Git Installation Guide](/cloud/guides/how-to-install-git-on-linux-mac-and-windows) - -## Installing Rook NFS on LKE - -1. Rook has several manifests on their [github repository](https://github.com/rook/nfs) that are used within this guide. Clone the [latest release](https://github.com/rook/rook/releases/) from the project's repository to your machine: - - ```command - git clone --single-branch --branch v1.7.3 https://github.com/rook/nfs.git - ``` - - {{< note >}} - If you do not want to use git, the raw manifest files can alternatively be obtained via wget or curl. - {{< /note >}} - -1. Apply the Rook operator and common manifests: - - ```command - kubectl create -f ~/nfs/cluster/examples/kubernetes/nfs/crds.yaml - kubectl create -f ~/nfs/cluster/examples/kubernetes/nfs/operator.yaml - ``` - -1. Validate that all pods are running as expected: - - ```command - kubectl -n rook-nfs-system get pod - ``` - - You Should see the following output: - - ```output - NAME READY STATUS RESTARTS AGE - rook-nfs-operator-5cc679885d-88769 1/1 Running 0 45h - ``` - -1. Overwrite the contents of the `webhook.yaml` file to the contents of the following file. This ensures that the webhook manifests will rely on a more up to date version of the Kubernetes API, ensuring that they're compatible with the latest versions of Kubernetes: - - ```file {title="/nfs/cluster/examples/kubernetes/nfs/webhook.yaml" lang=yaml} - apiVersion: v1 - kind: ServiceAccount - metadata: - name: rook-nfs-webhook - namespace: rook-nfs-system - --- - apiVersion: rbac.authorization.k8s.io/v1 - kind: Role - metadata: - name: rook-nfs-webhook - namespace: rook-nfs-system - rules: - - apiGroups: [""] - resources: ["secrets"] - resourceNames: - - "rook-nfs-webhook-cert" - verbs: ["get", "list", "watch"] - --- - apiVersion: rbac.authorization.k8s.io/v1 - kind: RoleBinding - metadata: - name: rook-nfs-webhook - namespace: rook-nfs-system - roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: rook-nfs-webhook - subjects: - - apiGroup: "" - kind: ServiceAccount - name: rook-nfs-webhook - namespace: rook-nfs-system - --- - apiVersion: cert-manager.io/v1 - kind: Certificate - metadata: - name: rook-nfs-webhook-cert - namespace: rook-nfs-system - spec: - dnsNames: - - rook-nfs-webhook.rook-nfs-system.svc - - rook-nfs-webhook.rook-nfs-system.svc.cluster.local - issuerRef: - kind: Issuer - name: rook-nfs-selfsigned-issuer - secretName: rook-nfs-webhook-cert - --- - apiVersion: cert-manager.io/v1 - kind: Issuer - metadata: - name: rook-nfs-selfsigned-issuer - namespace: rook-nfs-system - spec: - selfSigned: {} - --- - apiVersion: admissionregistration.k8s.io/v1 - kind: ValidatingWebhookConfiguration - metadata: - annotations: - cert-manager.io/inject-ca-from: rook-nfs-system/rook-nfs-webhook-cert - creationTimestamp: null - name: rook-nfs-validating-webhook-configuration - webhooks: - - clientConfig: - caBundle: Cg== - service: - name: rook-nfs-webhook - namespace: rook-nfs-system - path: /validate-nfs-rook-io-v1alpha1-nfsserver - failurePolicy: Fail - admissionReviewVersions: ["v1", "v1beta1"] - sideEffects: None - name: validation.nfsserver.nfs.rook.io - rules: - - apiGroups: - - nfs.rook.io - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - nfsservers - --- - kind: Service - apiVersion: v1 - metadata: - name: rook-nfs-webhook - namespace: rook-nfs-system - spec: - selector: - app: rook-nfs-webhook - ports: - - port: 443 - targetPort: webhook-server - --- - apiVersion: apps/v1 - kind: Deployment - metadata: - name: rook-nfs-webhook - namespace: rook-nfs-system - labels: - app: rook-nfs-webhook - spec: - replicas: 1 - selector: - matchLabels: - app: rook-nfs-webhook - template: - metadata: - labels: - app: rook-nfs-webhook - spec: - containers: - - name: rook-nfs-webhook - image: rook/nfs:v1.7.3 - imagePullPolicy: IfNotPresent - args: ["nfs", "webhook"] - ports: - - containerPort: 9443 - name: webhook-server - volumeMounts: - - mountPath: /tmp/k8s-webhook-server/serving-certs - name: cert - readOnly: true - volumes: - - name: cert - secret: - defaultMode: 420 - secretName: rook-nfs-webhook-cert - ``` - -1. Apply the Rook Admission Webhook and Cert Manager. Cert manager is a prerequisite for the webhook and applies an added layer of security: - - ```command - kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.8.0/cert-manager.yaml - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/webhook.yaml - ``` - -1. Verify that the webhook and cert manager were set up correctly: - - ```command - kubectl get -n cert-manager pod - kubectl -n rook-nfs-system get pod - ``` - - Output similar to the following should appear: - - ```output - kubectl get -n cert-manager pod - NAME READY STATUS RESTARTS AGE - cert-manager-cainjector-6d9776489b-wprdx 1/1 Running 0 45h - cert-manager-d7d8fb5c9-wv66w 1/1 Running 0 45h - cert-manager-webhook-544c44ccdc-stjjb 1/1 Running 0 45h - kubectl -n rook-nfs-system get pod - NAME READY STATUS RESTARTS AGE - rook-nfs-operator-5cc679885d-88769 1/1 Running 0 45h - rook-nfs-webhook-6ffb579d8c-wl59k 1/1 Running 0 45h - ``` - -1. There is an optional step in the Rook guide to add a Pod Security Policy. This is recommended for security. - - ```command - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/psp.yaml - ``` - -1. Create a service account user for Rook to run the NFS server with: - - ```command - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/rbac.yaml - ``` - -1. Edit the PVC portion of the default NFS server manifests in the `nfs.yaml` file before initializing the NFS server. The provided NFS manifest has two changes that need to be made. The first is the storage class for the PVC is left off, which assumes the default storage class. This can be explicitly defined to "linode-block-storage-retain" instead of assuming the default storage class. Secondly the `accessModes` is set to `ReadWriteMany` and the Linode block storage does not support `ReadWriteMany`. This should be changed to `ReadWriteOnce`. - - ```file {title="/nfs/cluster/examples/kubernetes/nfs/nfs.yaml" lang=yaml} - --- - apiVersion: v1 - kind: PersistentVolumeClaim - metadata: - name: nfs-default-claim - namespace: rook-nfs - spec: - storageClassName: linode-block-storage-retain # Add this line to specify the storage class to be used - accessModes: - - ReadWriteOnce # Edit this line to ReadWriteOnce - resources: - requests: - storage: 1Gi - --- - # Additionally this manifest contains the Rook NFSServer below the PVC - # This is not displayed as there are no modifications to that portion of the manifest - ``` - -1. Apply the updated nfs.yaml file, then add a Rook storage class that leverages the Rook NFS server. - - ```command - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/nfs.yaml - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/sc.yaml - ``` - -## Testing the Server - -To test the Rook NFS server, create two deployments that will leverage a singular storage class labeled as `rook-nfs-share1`. - -1. Deploy the PVC, busybox, and web-rc server using the following commands: - - ```command - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/pvc.yaml - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/busybox-rc.yaml - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/web-rc.yaml - ``` - -1. Create a service that will allow the busybox container to update the timestamp in the `index.html` file: - - ```command - kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/web-service.yaml - ``` - -The following command can now be entered to list the files being stored for in the volume for the busybox workload: - -```command -kubectl exec $(kubectl get pod -l app=nfs-demo,role=busybox -o jsonpath='{.items[0].metadata.name}') -- ls /mnt -``` - -This additional command can also be entered to list all of the files in the volume for the nginx workload: - -```command -kubectl exec $(kubectl get pod -l app=nfs-demo,role=web-frontend -o jsonpath='{.items[0].metadata.name}') -- ls /usr/share/nginx/html \ No newline at end of file diff --git a/docs/guides/kubernetes/setting-up-harbor-registry-with-lke/index.md b/docs/guides/kubernetes/setting-up-harbor-registry-with-lke/index.md deleted file mode 100644 index fd76687fb4e..00000000000 --- a/docs/guides/kubernetes/setting-up-harbor-registry-with-lke/index.md +++ /dev/null @@ -1,513 +0,0 @@ ---- -slug: setting-up-harbor-registry-with-lke -title: "Set Up a Self-Hosted Harbor Registry and Deploy Images to Kubernetes" -title_meta: "Install Harbor Registry and Deploy Images to Kubernetes" -description: "Learn the benefits of Harbor vs Docker Hub, how to install the Harbor registry software in the cloud, and how to connect a Kubernetes cluster to the registry." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2023-06-13 -keywords: ['what is harbor registry','install harbor kubernetes','harbor vs docker registry'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Harbor Docs: Harbor Installation and Configuration](https://goharbor.io/docs/2.7.0/install-config/)' -- '[Kubernetes: Pull an Image from a Private Registry](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/)' ---- - -Harbor is an open-source and self-hosted registry for container images. With Harbor, you can securely store and access your own collection of Docker images without resorting to an external service, like Docker Hub. Harbor's security and compliance features ensure your registry has the level of control that you need. - -## In this Guide - -This guide shows how to set up a Harbor registry on a dedicated compute instance in the cloud. It also shows how to create an LKE cluster that authenticates with the registry and pulls images from it. - -- The [What is Harbor](#what-is-harbor) section provides more information about the Harbor service and how it compares to other registries, like Docker Hub. - -- The [How to Set Up Harbor](#how-to-set-up-harbor) section details how to install and configure Harbor. - -- The [How to Start Working with Harbor](#how-to-start-working-with-harbor) section shows how to access the Harbor dashboard and how to add images to the registry. - -- The [How to Use the Harbor Registry with LKE](#how-to-use-the-harbor-registry-with-lke) section shows how to connect a Kubernetes cluster to the registry. - -## Before You Begin - -The Harbor installation in this guide assumes that you have [a domain name registered through a domain registrar](https://techdocs.akamai.com/cloud-computing/docs/getting-started-with-dns-manager#register-the-domain), and that you can edit the DNS records for this domain. This is so that SSL connections can be configured for the Harbor server. If you do not have a domain name, register one now. - -The infrastructure for this guide is created on the Akamai Cloud platform. If you do not already have one, [create an account](https://techdocs.akamai.com/cloud-computing/docs/getting-started) for the platform. - -The following is a summary of the infrastructure created in this guide. Instructions for creating these services are included later in the guide: - -- A compute instance that runs the Harbor server software - -- An LKE cluster with 3 nodes, used to demonstrate pulling an image from the Harbor server. - -If you would like to remove these services after you finish following the guide, review our [Stop Further Billing](https://techdocs.akamai.com/cloud-computing/docs/stop-further-billing) guide. For more information on how billing for services works, review the [Billing Overview](https://techdocs.akamai.com/cloud-computing/docs/understanding-how-billing-works). - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## What is Harbor? - -[Harbor](https://goharbor.io/) offers an open-source solution for self-hosting a container image registry. It is a project within the Cloud Native Computing Foundation (CNCF). Harbor puts security and compliance at the center of its design, with features like access policies and roles, vulnerability scanning, and image signing. - -With minimal configuration, Harbor integrates with tools like the Docker command-line interface (CLI) and kubectl. From the Docker CLI, you can log in to your Harbor registry to securely push and pull images. Kubernetes tools can likewise securely authenticate with your Harbor registry and allow you to deploy containers directly from images stored in the registry. - -### Harbor vs Docker Hub: Why Use Harbor? - -The most significant distinction between Harbor and Docker Hub is that Harbor can be self-hosted. By self-hosting Harbor, you gain a degree of control over things like compliance and access that you lack with a service like Docker Hub and other external cloud-hosted solutions. - -Harbor also comes with a suite of features that can enhance security and compliance. These features come with fine-grained controls for you to configure your registry. - -To help decide which registry to use, consider these trade-offs: - -- Docker Hub provides an accessible registry with plenty of existing images to leverage. However, you lose a degree of control over the data and you do not get the same robust security and compliance features. - -- Harbor centers on securing your images, with a suite of features to control access and roles and to mitigate vulnerabilities. Because it is self-hosted, Harbor has an initial setup and requires on-going system administration and maintenance. - -## How to Set Up Harbor - -To get started, you need to prepare a system to run the Harbor server software. The process involves installing and configuring some prerequisites and then installing Harbor itself. - -The server is your Harbor access point. It gives you an administrative interface for your Harbor instance as well as a host for your Harbor registry. Your Docker and Kubernetes tools can then connect to your Harbor registry to pull container images. - -### Configure DNS Records - -This tutorial requires you to have a domain for your Harbor server. Doing so allows you to use Let's Encrypt for SSL certification. For example code and commands, the tutorial uses `harbor.example.com` as the domain name. - -In your DNS manager for your own domain, create an [A record](/cloud/guides/dns-overview#a-and-aaaa) named `harbor`: - -- If you use the Linode DNS Manager, follow our [Manage DNS Records](https://techdocs.akamai.com/cloud-computing/docs/manage-dns-records) guide to create the new a record. - -- If you use another DNS manager, follow their instructions for creating the A record. - -{{< note >}} -If you use another DNS manager but would like to start using the Linode DNS Manager, refer to these guides: - - - [DNS Manager - Get Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started-with-dns-manager) - - [Configure Your Domain's Authoritative Name Servers](https://techdocs.akamai.com/cloud-computing/docs/configure-your-domains-authoritative-name-servers) - - [Incoming DNS Zone Transfers](https://techdocs.akamai.com/cloud-computing/docs/incoming-dns-zone-transfers) -{{< /note >}} - -### Provision a Compute Instance - -For this tutorial, the Harbor server is run on a compute instance. Follow these instructions to create the instance: - -1. Follow the [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guide to create the instance. Instructions for installing Harbor on Debian, Ubuntu, CentOS, and Fedora are included in this guide. - - Images stored with the registry are kept in the compute instance's included [persistent storage](https://techdocs.akamai.com/cloud-computing/docs/manage-disks-on-a-compute-instance#understanding-storage). Different instance plans have different amounts of included storage. You should pick a plan whose storage can accommodate the total size of the images you want to store. Review the [pricing page](/pricing/) to determine which plan has enough storage for you. If you intend to use this server for testing and learning, a plan with minimal specs is adequate. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system,create a limited user account, and harden SSH access. You may also wish to set the timezone and configure your hostname. - -### Install Docker and Docker Compose - -Docker and the Docker Compose plugin are required for running Harbor: - -1. Refer to the [official instructions](https://docs.docker.com/engine/install/#server) and install Docker Engine and the Docker Compose plugin for your compute instance's operating system. - -1. Ensure that the Docker daemon is running: - - ```command {title="Harbor compute instance"} - sudo systemctl status docker - ``` - - ```output - ● docker.service - Docker Application Container Engine - Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) - Active: active (running) since Thu 2023-02-16 20:38:07 UTC; 1s ago - Docs: https://docs.docker.com - ``` - -1. If it is not running (`active`), enable and start the Docker daemon with these commands. - - ```command {title="Harbor compute instance"} - sudo systemctl enable docker - sudo systemctl start docker - ``` - -### Configure the Firewall - -To allow connections to your Harbor server, your firewall needs to have the HTTP port (**80**) and HTTPS port (**443**) open. The steps below provide commands to open the HTTP and HTTPS ports on several major Linux distributions. These use the default firewall managers for each of the named distributions. - -{{< tabs >}} -{{< tab "Debian and Ubuntu" >}} -Use UFW to open the necessary ports: - -```command {title="Harbor compute instance"} -sudo ufw allow http -sudo ufw allow https -sudo ufw allow ssh -sudo ufw reload -``` - -Make sure UFW is enabled and running: - -```command {title="Harbor compute instance"} -sudo systemctl start ufw -sudo systemctl enable ufw -sudo ufw enable -``` -{{< /tab >}} -{{< tab "CentOS and Fedora" >}} -Use firewalld to open the necessary ports: - -```command {title="Harbor compute instance"} -sudo firewall-cmd --zone=public --add-service=http --permanent -sudo firewall-cmd --zone=public --add-service=https --permanent -sudo firewall-cmd --zone=public --add-service=ssh --permanent -sudo firewall-cmd --set-default-zone=public -sudo firewall-cmd --reload -``` - -Make sure firewalld is enabled and running: - -```command {title="Harbor compute instance"} -sudo systemctl start firewalld -sudo systemctl enable firewalld -``` -{{< /tab >}} -{{< /tabs >}} - -### Set Up SSL - -Using SSL lets you secure traffic to and from your Harbor server. You can create self-signed certificates, but typically it is more convenient to use a certificate authority like [Let's Encrypt](https://letsencrypt.org). - -[Certbot](https://certbot.eff.org) is a tool that provides an interface for requesting and downloading certificates from Let's Encrypt. This tutorial uses Certbot to set up an SSL certificate: - -1. To install Certbot, follow one of our [Certbot for NGINX guides](/cloud/guides/enabling-https-using-certbot#nginx). For this tutorial, follow just the **Installing Snapd** and **Installing Certbot** sections. *Do not* follow that guide's steps for requesting a certificate. - -1. Use Certbot to request a certificate for your domain (given after the `-d` option), using the [standalone verification](https://eff-certbot.readthedocs.io/en/stable/using.html#standalone) option. Replace the domain in this command with your Harbor server's domain: - - ```command {title="Harbor compute instance"} - sudo certbot certonly --standalone -d harbor.example.com - ``` - -1. Once the process has finished, Certbot should output the location of your certificate and key files, as shown here. These locations need to be specified in the Harbor configuration file later in this guide, so copy them into your notes now. - - ```output - [...] - Successfully received certificate. - Certificate is saved at: /etc/letsencrypt/live/harbor.example.com/fullchain.pem - Key is saved at: /etc/letsencrypt/live/harbor.example.com/privkey.pem - [...] - ``` - -### Install Harbor - -1. Download the package with the latest Harbor installer from the Harbor [releases page](https://github.com/goharbor/harbor/releases/latest). You can choose either the online or offline installer. - - The command below streamlines this step. It grabs the download link for the latest offline installer package and downloads it as `harbor-offline-installer.tgz`. - - ```command {title="Harbor compute instance"} - curl -s https://api.github.com/repos/goharbor/harbor/releases/latest \ - | grep "browser_download_url.*harbor-offline-installer.*.tgz\"" \ - | tail -n 1 \ - | cut -d : -f 2,3 \ - | tr -d \" \ - | wget -O harbor-offline-installer.tgz -qi - - ``` - - Be sure to do this in a directory where you want to keep the Harbor installer's own subdirectory. You should retain the Harbor installer even after installation, as it includes scripts for making configuration changes later. - -1. Extract the installer package, which places the installer contents within a `harbor/` subdirectory of the current directory. You may first need to install the `tar` tool, which you should be able to get through your system's package manager. - - ```command {title="Harbor compute instance"} - tar xzvf harbor-offline-installer.tgz - ``` - -1. Navigate into the new `harbor/` directory and make a copy of the included configuration file template, named `harbor.yml`. This copy serves as your Harbor instance configuration. - - ```command {title="Harbor compute instance"} - cd harbor - cp harbor.yml.tmpl harbor.yml - ``` - -1. Open the `harbor.yml` file in a text editor. Begin adjusting the configuration to fit your needs. - - Several of the required parameters can retain their default values, like the `http` parameter. The following are required parameters that you should make initial adjustments to: - - - `hostname`: Set this to your Harbor server's domain (e.g. `harbor.example.com`). - - - `https`: Modify the `certificate` and `private_key` sub-parameters to point to the locations of your SSL certificate and key files. These were provided by Certbot in the [Set Up SSL](#set-up-ssl) section. - - Using the example output in the previous section, the parameters would look like the following: - - ```file {title="harbor.yml" lang="yml"} - # [...] - https: - port: 443 - certificate: /etc/letsencrypt/live/harbor.example.com/fullchain.pem - private_key: /etc/letsencrypt/live/harbor.example.com/privkey.pem - # [...] - ``` - - As well, adjust the following passwords in the configuration file. If these are not changed, your Harbor instance is more vulnerable to attacks: - - - `harbor_admin_password`: Replace with an initial password for the administrator user. This password is reset again in the next section after first logging in, but changing the default here mitigates attacks between when you install the server and when you log in. - - - `database`: Replace the `password` sub-parameter with a more secure password for Harbor's PostgreSQL database. - -1. With your configuration changes in place, run the installation script: - - ```command {title="Harbor compute instance"} - sudo ./install.sh - ``` - - You should be able to follow along with the progress as Harbor installs. At the end the script notifies you that the installation has completed successfully: - - ```output - ✔ ----Harbor has been installed and started successfully.---- - ``` - -## How to Start Working with Harbor - -Once the installation has finished, Harbor begins running and is ready to use. The next sections show how to access your Harbor server and how to populate the registry with a custom Docker image. - -### Access Harbor - -Harbor's dashboard is exposed at the domain you configured above. For this tutorial, that means navigating to `https://harbor.example.com` to access the Harbor login page. - -[![The Harbor login page](harbor-login_small.png)](harbor-login.png) - -Enter the username `admin`, and the administrator password from the `harbor.yml` file from the previous section. - -Logging in takes you to the Harbor dashboard, where you can see details of your Harbor instance and begin managing its settings. Be sure to take a look around and familiarize yourself with all of the options available. - -[![The Harbor dashboard](harbor-dashboard_small.png)](harbor-dashboard.png) - -You should change your administrator password after you first log in: - -1. Select the **admin** option from the upper right - -1. Select the **Change Password** from the dropdown menu - -### Add Images to Harbor - -From the Harbor dashboard, you can see all of your Harbor resources. To get the most out of Harbor, you should populate the registry with container images. These images then appear within the dashboard and are accessible directly from your Harbor registry's address. - -Harbor organizes images into projects, so you should first either create a project to your liking or modify the default `library` project. You can do all of this from the **Projects** option on the left menu. - -The examples in this tutorial use the default project (`library`) but change the project's accessibility from public to private: - -1. Select **Projects** from the left menu. - -1. Select **library** from the list of projects. - -1. Select the **Configuration** tab. - -1. Uncheck the **Public** option. - -1. Click the **Save** button. - -The next sections show how to create your own Docker image and push it to your Harbor registry. - -#### Create a Docker Image - -To begin, you need a Docker image. This tutorial creates a simple image that includes NGINX and a basic website. Later in the tutorial, you can see this website fully deployed from Harbor to your LKE cluster. - -The following instructions can be run from your local workstation. They require that you have Docker installed on your workstation. If you do not have Docker installed locally, review the [official instructions](https://docs.docker.com/engine/install/) for installing it. - -1. Make a directory to store the image configuration file and its resources, and navigate into that directory. - - ```command {title="Your workstation"} - mkdir ~/example-nginx-image - cd ~/example-nginx-image - ``` - -1. Add an `nginx-default.conf` file there with the contents shown here. These define a minimal NGINX static server. - - ```file {title="nginx-default.conf" lang="conf"} - server { - listen 80; - listen [::]:80; - server_name localhost; - location / { - root /usr/share/nginx/html; - index index.html index.htm; - } - error_page 500 502 503 504 /50x.html; - location = /50x.html { - root /usr/share/nginx/html; - } - } - ``` - -1. Make an `html/` subdirectory where the static site content can be stored. - - ```command {title="Your workstation"} - mkdir html - ``` - -1. Add an `index.html` file into the `html/` directory with the file the contents shown here. - - ```file {title="html/index.html" lang="html"} - - - - - Example Nginx Website - - -

An Example Nginx Website

-

Welcome to your own Nginx website!

- - - ``` - -1. Add a `Dockerfile` to the image directory with the file the contents shown here. This file defines your Docker image. - - ```file {title="Dockerfile" lang="docker"} - FROM nginx:latest - RUN rm /etc/nginx/conf.d/default.conf - COPY nginx-default.conf /etc/nginx/conf.d/default.conf - COPY html /usr/share/nginx/html - ``` - -1. Use the Docker `build` command to build an image from the Dockerfile. This example gives the image the name `example-nginx-image`. - - ```command {title="Your workstation"} - sudo docker build -t example-nginx-image . - ``` - -1. You can now verify that the image has been successfully created by listing your Docker images: - - ```command {title="Your workstation"} - sudo docker images - ``` - - ```output - REPOSITORY TAG IMAGE ID CREATED SIZE - example-nginx-image latest 123658fe84af 5 seconds ago 142MB - ``` - -{{< note >}} -You can learn more about building Docker images through our guide on [How to Use a Dockerfile to Build a Docker Image](/cloud/guides/how-to-use-dockerfiles). -{{< /note >}} - -#### Push an Image to Harbor - -With a Docker image built and ready, you can use the Docker CLI to push that image to your Harbor registry: - -1. Because the Harbor project is private, you need to log in to the registry from the Docker CLI: - - ```command {title="Your workstation"} - sudo docker login harbor.example.com - ``` - - This prompts you for a username and password. Use the `admin` credentials used to log in to the Harbor interface (use the new password you set after logging in for the first time). - -1. Add a tag to your Docker image. In the example below, the image is tagged with a version number `1.0.0`. Notice that the image is tagged for the `library` project within the Harbor registry. - - In this command, replace `harbor.example.com` with your Harbor server's domain. - - ```command {title="Your workstation"} - sudo docker tag example-nginx-image harbor.example.com/library/example-nginx-image:1.0.0 - ``` - -1. Push the Docker image up to your Harbor registry. In this command, replace `harbor.example.com` with your Harbor server's domain. - - ```command {title="Your workstation"} - sudo docker push harbor.example.com/library/example-nginx-image:1.0.0 - ``` - - ```output - The push refers to repository [harbor.example.com/library/example-nginx-image] - [...] - 1.0.0: digest: sha256:f37ae9e729f7c3a0d5f8e84e8dcfc019093ac76b48e885568c8c5b6b163d5b43 size: 2191 - ``` - -You can go to the Harbor dashboard in your browser to see the results. Navigate to the **Projects** page using the menu on the left, and select the **library** project from the listing. You should see your image listed. - -[![An image shown in the Harbor registry dashboard](harbor-project_small.png)](harbor-project.png) - -## How to Use the Harbor Registry with LKE - -Harbor can act as the image registry for a Kubernetes cluster. The steps in this section show how to deploy the example Docker image in the previous section from your Harbor registry to your LKE cluster. - -1. Provision an LKE cluster. Follow our [Linode Kubernetes Engine - Get Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started-with-lke-linode-kubernetes-engine) guide to install the kubectl tool on your local workstation, deploy a cluster, and download your cluster's kubeconfig. The new cluster should have a node pool with three nodes. The nodes can have minimal specifications, because they are only used to serve a small website. - - The commands in the next steps should be run from your workstation. - -1. Create a secret with kubectl. This secret allows your cluster to have an authenticated connection to your private Harbor registry. - - Replace the `--docker-server` URL with your Harbor server's URL. Replace the `--docker-password` value with the credentials for your Harbor server. - - ```command {title="Your workstation"} - kubectl create secret docker-registry harbor-registry-secret --docker-server="https://harbor.example.com" --docker-username="admin" --docker-password="Harbor12345" - ``` - -1. Create an `example-nginx-deployment.yml` file with the contents shown here. This configuration deploys the `example-nginx-image` custom image alongside a service for accessing the containers. - - This deployment uses three `replicas`. You can adjust that number to match the number of nodes in your LKE cluster on **line 18**. - - **Line 29** refers to the URL of your image on the Harbor server. Update the domain for this URL to match your Harbor server's domain. - - ```file {title="example-nginx-deployment.yml" lang="yml" hl_lines="18 29"} - apiVersion: v1 - kind: Service - metadata: - name: example-nginx-service - spec: - type: NodePort - selector: - app: example-nginx - ports: - - port: 80 - targetPort: 80 - --- - apiVersion: apps/v1 - kind: Deployment - metadata: - name: example-nginx-app - spec: - replicas: 3 - selector: - matchLabels: - app: example-nginx - template: - metadata: - labels: - app: example-nginx - spec: - containers: - - name: example-nginx-instance - image: harbor.example.com/library/example-nginx-image:1.0.0 - imagePullPolicy: Always - ports: - - containerPort: 80 - imagePullSecrets: - - name: harbor-registry-secret - ``` - -1. Apply the configuration file to your cluster: - - ```command {title="Your workstation"} - kubectl apply -f example-nginx-deployment.yml - ``` - -1. Verify that the deployment has completed successfully by listing the pods. You should see three, each with a `Running` status. - - ```command {title="Your workstation"} - kubectl get pods - ``` - - ```output - NAME READY STATUS RESTARTS AGE - example-nginx-app-646f4d5b6-8j8xc 1/1 Running 0 1m - example-nginx-app-646f4d5b6-gnlbk 1/1 Running 0 1m - example-nginx-app-646f4d5b6-pdr24 1/1 Running 0 1m - ``` - -1. Check to see that the deployed application is performing as expected. The custom Docker image serves a static web page, so you can visit the web page to see that it is working. - - You can use kubectl to forward port `80` from within the cluster. Forward this traffic to port `8001` on your local workstation: - - ```command {title="Your workstation"} - kubectl port-forward service/example-nginx-service 8001:80 - ``` - - Navigate to `localhost:8001` in a web browser to see the deployed application. - -## Conclusion - -You now have an operational Harbor registry. You have also authenticated your Docker CLI and a Kubernetes cluster with the registry. Between these tools, you have all you need to start managing a full registry and including it into your infrastructure. - -To learn more about managing your Harbor registry and about the configuration features, be sure to refer to the Harbor documentation linked below. The Harbor dashboard gives you extensive control of access policies and roles, vulnerability scanning, and other features to secure your registry. diff --git a/docs/guides/kubernetes/using-octant-with-kubernetes-a-tutorial/index.md b/docs/guides/kubernetes/using-octant-with-kubernetes-a-tutorial/index.md deleted file mode 100644 index 5aafe7368cb..00000000000 --- a/docs/guides/kubernetes/using-octant-with-kubernetes-a-tutorial/index.md +++ /dev/null @@ -1,325 +0,0 @@ ---- -slug: using-octant-with-kubernetes-a-tutorial -title: "A Overview of Using Octant with Kubernetes" -title_meta: "How to Use Octant with Kubernetes (A Tutorial)" -description: "Octant is a visual web-based dashboard for Kubernetes clusters. This guide explores Octant's interface and how it makes troubleshooting Kubernetes easier." -authors: ["Linode"] -contributors: ["Linode"] -published: 2019-11-06 -keywords: ['octant','kubernetes','cluster','graph','dashboard','troubleshoot'] -tags: ["monitoring","kubernetes","web application"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: using-octant-with-kube.jpg -external_resources: -- '[Octant](https://octant.dev/)' -- '[Octant on GitHub](https://github.com/vmware-tanzu/octant)' -- '[Octant on Twitter](https://twitter.com/projectoctant)' -aliases: [] ---- - -## What is Octant? - -Octant is a web application that talks to your Kubernetes clusters and provides an easy-to-read dashboard for the objects in your clusters. A marquee feature of Octant is its ability to show your objects and their relations in a graph format: - -![Octant Resource Viewer Graph Example](octant-resource-viewer-graph-intro-example.png) - -Octant aims to be developer-centric. It is designed to help application developers (who may not be familiar with Kubernetes) better understand how their applications are deployed and troubleshoot issues when they arise. - -### In This Guide - -This guide will explore a few ways to use Octant with some example software deployments: - -- You'll see how to [navigate Octant's main interfaces](#navigating-the-interface) after a Helm chart has been installed on a cluster. - -- A [troubleshooting thought experiment](#troubleshooting-with-octant) will show how Octant can make discovering issues in your cluster easier. - -{{< note >}} -This guide assumes familiarity with the Kubernetes concepts outlined in Linode's [Beginner's Guide to Kubernetes](/cloud/guides/beginners-guide-to-kubernetes). If you have never set up a Kubernetes cluster before, it's also recommended that you do so to get the most out of this guide. The [How to Deploy Kubernetes on Linode with the k8s-alpha CLI](/cloud/guides/how-to-deploy-kubernetes-on-linode-with-k8s-alpha-cli) guide describes a one-line command for generating a cluster. -{{< /note >}} - -## Before you Begin - -The examples in this guide have been tested on a cluster running with Linode with Linode's [CCM](/cloud/guides/kubernetes-reference#linode-cloud-controller-manager) and [CSI](/cloud/guides/kubernetes-reference#container-storage-interface) plugins installed. If you would like to install these examples as well, a cluster made with [k8s-alpha CLI](/cloud/guides/how-to-deploy-kubernetes-on-linode-with-k8s-alpha-cli) will meet these criteria. - -{{< note type="alert" >}} -These examples will create billable services. To stop billing for these services after reading the guide, be sure to read the tear-down instructions at the end of each section. If you created a new cluster for this guide, you can remove the cluster's Nodes from the [Linode Cloud Manager](https://cloud.linode.com). - -If you remove the resources afterward, you will only be billed for the hour(s) that the resources were present on your account. Consult the [Billing and Payments](https://techdocs.akamai.com/cloud-computing/docs/understanding-how-billing-works) guide for detailed information about how hourly billing works. [Linode's pricing page](https://www.linode.com/pricing/) lists the rate for each Linode service. -{{< /note >}} - -## Installing Octant - -Octant does not run in your cluster. Instead, it connects to your cluster remotely and observes activity within the cluster. It also does not require special privileges to run--Octant will use the same context information that you use for kubectl. This also means that users with restricted access to a cluster can run Octant; Octant will just display the objects that are visible to that user. - -Octant can be run as a local server process on your workstation. To install and run it: - -1. Install from a package available on your operating system: - - - **Linux**: download a `.deb` or `.rpm` from the [releases page](https://github.com/vmware-tanzu/octant/releases) on GitHub, then install it with `dpkg -i` or `rpm -i`. For example: - - ```command - wget https://github.com/vmware-tanzu/octant/releases/download/v0.9.1/octant_0.9.1_Linux-64bit.deb - dpkg -i octant_0.9.1_Linux-64bit.deb - `` - - - **macOS**, with Homebrew: - - ```command - brew install octant - ``` - - - **Windows**, with Chocolatey: - - ```command - choco install octant - ``` - - {{< note >}} - Alternative installation instructions are available in the project's [README](https://github.com/vmware-tanzu/octant#installation) on GitHub. - {{< /note >}} - -1. Then, start the server. In your terminal, run: - - ```command - octant - ``` - -1. If it starts successfully, you should see a message similar to: - - ```output - ... - - Dashboard is available at http://127.0.0.1:7777 - ``` - - {{< note >}} - If it does not start successfully, check that you can connect to your cluster with kubectl. For example, try running: - - ```command - kubectl get pods - ``` - {{< /note >}} - -1. The dashboard may load automatically in your browser, or you can load the dashboard address (e.g. `http://127.0.0.1:7777`) in your browser if it does not. You should see the Octant dashboard. - -## Navigating the Interface - -The interfaces that Octant provides are meant to be a complement, and not a replacement, for kubectl. When using Octant, you may find that you sometimes need to return to kubectl to perform certain actions. Still, the Octant dashboard will serve as a helpful overview when inspecting your cluster. - -{{< note >}} -The cluster objects visible in the following screenshots were created by installing the [Helm chart](https://github.com/helm/charts/tree/master/stable/ghost) for the [Ghost](https://ghost.org) blogging software. The [How to Install Apps on Kubernetes with Helm 2](/cloud/guides/how-to-install-apps-on-kubernetes-with-helm-2) guide outlines how to install this software. Please note that this guide uses Helm 2 and not Helm 3 to install the software. -{{< /note >}} - -- When first viewing the dashboard, a list of all of your cluster objects will be shown: - - ![Octant Cluster Overview](octant-cluster-overview.png) - - The left navigation will display a hierarchy of the objects that can be viewed, and the right column will display your objects. In the right corner of the top navigation, you can quickly switch between your workstation's cluster contexts. The top navigation also allows you to change between cluster namespaces, and to filter your objects by label. - -- Consider the Services item in the left navigation. When clicked on, only your Services will appear in a table to the right: - - ![Octant Services View](octant-services.png) - -- Important attributes of your Services will be listed in the columns of this table. In particular, the labels are buttons which can be clicked on: - - ![Octant Services View - Labels Highlighted](octant-services-labels-highlighted.png) - - If you click on the **release:my-blog** button, all Services without this label will be hidden: - - ![Octant Services View - release:my-blog](octant-services-release-my-blog.png) - - This view shows that the Ghost Helm chart set up two Services: one for the Ghost front-end, and one for its database. - - {{< note >}} - Multiple labels can be selected at the same time. - {{< /note >}} - -- This filter will persist across all other views. For example, if you navigate to the Pods view, only Pods with the `release:my-blog` label will be shown. To clear your filters, click the **clear filters** link under the filter dropdown in the top navigation: - - ![Octant Filter Menu - Clear Filters](octant-clear-filters.png) - -### Inspecting an Object - -Clicking on an object will show more detail for that object, including the visual graph view for that object's relationships. - -- For example, the detail view for the Ghost front-end service shows a summary with **Configuration**, **Status**, **Metadata**, **Endpoints**, and **Events** panels: - - ![Octant Service Detail View - Ghost front-end](octant-service-my-blog-ghost-summary.png) - - - The **Configuration** panel makes it easy to see which selector the Service uses to identify your Pods. This can also be directly edited (via the **Edit** link in the panel). Octant's development roadmap includes adding more direct-editing features like this. - - - The **Status** panel shows the external IP address for the Service, because it was implemented with type `LoadBalancer`. If you view this IP in your browser, you will see the Ghost blog. - - - The **Endpoints** tab shows the Service's Pods. - - - The **Events** tab shows the history of events for the Service. This information can be useful when troubleshooting. - - {{< note >}} - Other panels will appear for different object types. For example, Pods will show container environment variables, Volume mounts, node resources requests and limits, and taints, among other information. - {{< /note >}} - -- The **Resource Viewer** tab will reveal the object relationship graph for this Service: - - ![Octant Service Detail View - Ghost front-end](octant-service-my-blog-ghost-resource-viewer.png) - - This view will display color-coded cells for each object, indicating the object's status. The presence or absence of objects in this graph can be helpful when troubleshooting. For example, if a Service's graph does not show connections to a Pod, then the Service may not be configured to use the right Pod selector. - -- The third **YAML** tab shows the YAML representation for the object. - - {{< note >}} - The detail view for Pods will also show a fourth **Logs** tab,which will follow and display the logs for a Pod in real-time: - - ![Octant Pod Detail View - Ghost front-end logs](octant-pod-my-blog-ghost-logs.png) - {{< /note >}} - -### Navigation Example Tear-Down - -To delete the objects created by the Helm chart used in this section, run: - -```command -helm delete -``` - -The Helm release name for can be determined by running: - -```command -helm ls -``` - -## Troubleshooting with Octant - -To get a better sense of how Octant can assist with troubleshooting software deployments on Kubernetes, consider this scenario: - -Your colleague Nathan is learning Kubernetes, and he's deploying a "Hello World" Node.js application as a way of testing his knowledge. You offer to install the application on your cluster to follow along with his work. - -### Install the Node.js Application - -1. Nathan has uploaded a Docker image for the application to Docker Hub under the name `linodedocs/kubernetes_using-octant-with-kubernetes-a-tutorial_hello-world:v1`. He also tells you that the Kubernetes manifest for the application is hosted here: [release-1.0.yaml](release-1.0.yaml). Download the file and apply it to your cluster with kubectl: - - ```command - kubectl apply -f release-1.0.yaml - ``` - - Your terminal should respond with: - - ```output - service/hello-world-service created - deployment.apps/hello-world-deployment created - ``` - - {{< note >}} - The Dockerfile and other files used to create the Docker Hub image are located [here](hello-world-node-js-1.tar.gz). Inspecting these files is not necessary for the tutorial. - {{< /note >}} - -1. To learn about how the application is structured on your cluster, you view it in Octant. When visiting the Services tab, you find the new `hello-world-service` object: - - ![Octant Services - Hello World with External IP Highlighted](octant-services-table-with-hello-world-1_0.png) - -1. The external IP address for the Service will also be shown in the new table entry (highlighted above). Visiting this address in your browser will return the "Hello World" message as expected. - -1. If you click on the `hello-world-service` Service and then navigate to the **Resource Viewer** tab, the relationships for it will be shown. If you click on the Pods cell, a right-hand navigation will appear. The panel in this navigation will show three green dots: - - ![Octant Service Resource Viewer - Hello World 1.0](octant-service-hello-world-1_0-resource-viewer.png) - - The three green dots indicate that three Pods were created that match the selector for the Service. Clicking on each will navigate to that Pod's detail view. - -### Update the Node.js Application - -1. Nathan tells you that he has updated the application to return "Hello Octant" instead of "Hello World", and he notes that the new Docker Hub image is named `linodedocs/kubernetes_using-octant-with-kubernetes-a-tutorial_hello-world:v2`. The new Kubernetes manifest is here: [release-2.0.yaml](release-2.0.yaml). Download the manifest and apply it to your cluster with kubectl: - - ```command - kubectl apply -f release-2.0.yaml - ``` - - Your terminal should respond with: - - ```output - service/hello-world-service unchanged - deployment.apps/hello-world-deployment configured - ``` - - {{< note >}} - The Dockerfile and other files used to create the Docker Hub image are located [here](hello-world-node-js-2.tar.gz). Inspecting these files is not necessary for the tutorial. - {{< /note >}} - -1. If you visit the external IP for the application in your browser again, it will still display the same "Hello World" message. If you return to the Resource Viewer graph for the `hello-world-service` Service in Octant, you'll see that it has been updated: - - ![Octant Service Resource Viewer - Hello World 2.0](octant-service-hello-world-2_0-resource-viewer.png) - -1. The Deployment has created a new ReplicaSet to run the updated application under. The orange color for the Deployment, ReplicaSet, and Pods indicates an issue with the update that will need further investigating. - - {{< note >}} - The Deployment keeps the older ReplicaSet and Pods running in place until the new ReplicaSet is healthy. - {{< /note >}} - -1. To investigate why the new Pod is unhealthy, click on the orange Pod cell in the graph, and then click on the orange dot in the right-hand panel that appears (highlighted above). - -1. The detail view for the Pod will appear. Scroll down to the **Events** table at the bottom. A `Failed to pull image "linodedocs/kubernetes_using-octant-with-kubernetes-a-tutorial_hello-world:v2a"` message should appear in the table: - - ![Octant Pod Events - Hello World 2.0](octant-pod-hello-world-2_0-events.png) - -1. The name for the Pod's image has a typo and should be corrected. On your workstation, open the release-2.0.yaml manifest and update **line 32** so that it refers to `linodedocs/kubernetes_using-octant-with-kubernetes-a-tutorial_hello-world:v2` instead of `linodedocs/kubernetes_using-octant-with-kubernetes-a-tutorial_hello-world:v2a`. Save the file, then apply the change to your cluster: - - ```command - kubectl apply -f release-2.0.yaml - ``` - -1. If you visit the external IP for the application in your browser again, it will still display the same "Hello World" message. If you return to the Resource Viewer graph for the `hello-world-service` Service in Octant, you'll see that it has been updated again: - - ![Octant Service Resource Viewer - Hello World 2.0 (after manifest update)](octant-service-hello-world-2_0-resource-viewer-after-manifest-update.png) - -1. The new Pod is no longer colored orange, but the ReplicaSet still is, indicating that other problems also need to be fixed. Click on the orange ReplicaSet cell in the graph, and then click on the title of the right-hand panel that appears (highlighted above). - -1. The detail view for the ReplicaSet will appear. The Pods panel on this page will show that the Pod has restarted several times: - - ![Octant ReplicaSet Pods - Hello World 2.0](octant-replicaset-hello-world-2_0-pods.png) - -1. Clicking on the Pod's name in this panel will take you to the detail view for it. This **Events** panel in this view will show the recent restarts. A good next step would be to investigate the logs for the Pod, but you may find that the **Logs** tab shows no content. In this case, it's worth switching out to kubectl to try and get logs from previous restarts: - - ```command - kubectl logs hello-world-deployment-7b69c98754-f4zk5 --previous=true - ``` - - {{< note >}} - Replace the Pod name above with the name of yours. - {{< /note >}} - - ```output - /usr/src/app/server.js:7 - res.send('Hello Octant); - ^^^^^^^^^^^^^^^ - - SyntaxError: Invalid or unexpected token - at Module._compile (internal/modules/cjs/loader.js:723:23) - at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10) - at Module.load (internal/modules/cjs/loader.js:653:32) - at tryModuleLoad (internal/modules/cjs/loader.js:593:12) - at Function.Module._load (internal/modules/cjs/loader.js:585:3) - at Function.Module.runMain (internal/modules/cjs/loader.js:831:12) - at startup (internal/bootstrap/node.js:283:19) - at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3) - ``` - -1. Nathan's code contains the missing quote syntax error reported in these logs, and you let him know what should be fixed. He responds that he's uploaded a `linodedocs/kubernetes_using-octant-with-kubernetes-a-tutorial_hello-world:v3` image to Docker Hub and tells you to apply his updated manifest here: [release-3.0.yaml](release-3.0.yaml). Download the file and apply it to your cluster with kubectl: - - ```command - kubectl apply -f release-3.0.yaml - ``` - - {{< note >}} - The Dockerfile and other files used to create the Docker Hub image are located [here](hello-world-node-js-3.tar.gz). Inspecting these files is not necessary for the tutorial. - {{< /note >}} - -1. If you return to the external IP for the application in your browser, it should now display "Hello Octant". The cells in Octant's Resource Viewer graph should all be colored green as well. - -### Troubleshooting Example Tear-Down - -To delete the objects created by the troubleshooting example, run: - -```command -kubectl delete -f release-3.0.yaml -``` - -## Next Steps - -Octant also provides a robust plugin functionality. Plugins help Octant track cluster resources that are not part of Octant's core codebase, and Octant can show content from a plugin in-line in your dashboard alongside other standard interface elements. Review [Octant's documentation](https://octant.dev/docs/master/plugins/) for more information on plugins. \ No newline at end of file diff --git a/docs/guides/networking/linode-network/set-up-an-ipv6-tunnel-on-your-linode/index.md b/docs/guides/networking/linode-network/set-up-an-ipv6-tunnel-on-your-linode/index.md deleted file mode 100644 index 3c8ba803e76..00000000000 --- a/docs/guides/networking/linode-network/set-up-an-ipv6-tunnel-on-your-linode/index.md +++ /dev/null @@ -1,210 +0,0 @@ ---- -slug: set-up-an-ipv6-tunnel-on-your-linode -title: Set Up an IPv6 Tunnel on Your Linode -description: 'This guide provides you with instructions on how to setup an IPv6 tunnel to connect your system to a network running on IPv4 and pass traffic from one to another.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2011-04-29 -modified: 2018-12-11 -keywords: ["ipv6", "tunnel", "broker", "networking"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -tags: ["networking","linode platform"] ---- - -![Set up an IPv6 Tunnel on Your Linode](Set_Up_an_IPv6_Tunnel_on_Your_Linode_smg.jpg) - -As IPv4 address exhaustion nears, many people are making the switch to IPv6. Linode offers [native IPv6](https://techdocs.akamai.com/cloud-computing/docs/an-overview-of-ipv6-on-linode) addresses in all locations. - -An IPv6 tunnel lets a system reach an IPv6 network using existing IPv4 connectivity. Follow this guide if you: - - - Have a tunnel that you are migrating to your Linode. - - - Don't have IPv6 from your ISP, and want to interact with your Linode through its IPv6 address. By configuring the tunnel to a local computer instead of your Linode, you can connect to your Linode via the IPv6 address. - -## Before You Begin - -You should have already signed up for an IPv6 tunnel through a tunnel broker. [Wikipedia contains a list of tunnel brokers by region](http://en.wikipedia.org/wiki/List_of_IPv6_tunnel_brokers), and we encourage you to research each before you decide which to use. The steps outlined in this guide were performed using tunnels from Hurricane Electric (HE). - -## General Setup - -Once you have signed up for a tunnel, you will need to issue a few commands on your Linode. Arch and Gentoo Linux users may need to install the `iproute2` package before continuing. - -{{< note >}} -Configuration of an IP tunnel using this method will not be persistent after reboot and will need to be reconfigured after restarting your Linode. -{{< /note >}} - -1. Because some of these steps will temporarily disable networking on the Linode, begin by logging in using either the [Lish](https://techdocs.akamai.com/cloud-computing/docs/access-your-system-console-using-lish) or [Glish](https://techdocs.akamai.com/cloud-computing/docs/access-your-desktop-environment-using-glish) interface. - -2. Use the `ip` tool to add the tunnel device. Ours is called `he-ipv6` to match the device described in Hurricane Electric's examples. Replace `203.0.113.10` with the endpoint of your tunnel, and `198.51.100.5` with your Linode's IP address. The information for the endpoint can be found in your tunnel broker's web interface, and your Linode's IP address can be found under the [Networking](https://techdocs.akamai.com/cloud-computing/docs/managing-ip-addresses-on-a-compute-instance#viewing-ip-addresses) tab of the Linode Cloud Manager: - - ip tunnel add he-ipv6 mode sit remote 203.0.113.10 local 198.51.100.5 ttl 255 - ip link set he-ipv6 up - - For more information on how the `ip` tool configures tunnels, see the [ip-tunnel documentation](http://man7.org/linux/man-pages/man8/ip-tunnel.8.html). - -3. Assign IPv6 address and routing information to your new tunnel device. Replace `2001:db8:1234:5678::2/64` with the IPv6 address assigned to you. This information should be provided to you by your tunnel broker as your "Client IPv6 Address": - - {{< note type="alert" respectIndent=false >}} -The `ifdown` command **will halt all network traffic to your Linode**. This step is included to avoid an error when adding the IPv6 route. It may not be required on all Linux distributions. - -On Arch Linux, replace the `ifdown` and `ifup` commands with `ip link set eth0 down` and `ip link set eth0 up` -{{< /note >}} - - ip addr add 2001:db8:1234:5678::2/64 dev he-ipv6 - ifdown eth0 - ip route add ::/0 dev he-ipv6 - ifup eth0 - ip -f inet6 addr - - The final command in Line 5 will show all devices with IPv6 addresses, and should have a block similar to this: - - 13: he-ipv6@NONE: mtu 1480 state UNKNOWN qlen 1 - inet6 2001:db8:1234:5678::2/64 scope global - valid_lft forever preferred_lft forever - inet6 fe80::0000:0000/64 scope link - valid_lft forever preferred_lft forever - -4. Test the tunnel. Replace `he-ipv6` with the name of your tunnel. On Arch Linux and other distributions without `ping6`, use `ping -6` instead. - - ping6 -I he-ipv6 irc6.oftc.net - - {{< note respectIndent=false >}} -Your system must allow ICMPv6 in through the firewall for pings to be returned. -{{< /note >}} - -If everything is working, you should see ping replies. If not, go back and make sure that you haven't made any errors. - -## Manual Configuration - -The instructions in this section will allow you to manually configure your IPv6 tunnel. This can permanently affect your connectivity across reboots. Read the IPv6 documentation for your distribution before proceeding. -{{< note type="alert" >}} -When manually modifying your network configuration, always disable [Network Helper](https://techdocs.akamai.com/cloud-computing/docs/automatically-configure-networking#individual-compute-instance-setting) first to avoid having your changes overwritten on reboot. -{{< /note >}} - -### Debian and Ubuntu - -Debian and Ubuntu users (versions before Ubuntu 16.04 are not covered here) can perform the following steps to set up a tunnel on their Linode. - -1. In the following excerpt, replace the: - - * `address` value with the "Client IPv6 address." - * `gateway` value with the "Server IPv6 address." - * `endpoint` value with the endpoint, or "Server IPv4 Address," that your tunnel broker provides you. Generally this endpoint is in a geographical location that is close to your Linode. - * `local` value with your Linode's IP address. If you have multiple IPs, make sure that this IP is set to the same address as the one you used to sign up for the tunnel. - - Insert the following into your `/etc/network/interfaces` file: - - {{< file "/etc/network/interfaces" >}} -auto he-ipv6 -iface he-ipv6 inet6 v4tunnel - address 2001:db8:1234:5678::2 - netmask 64 - endpoint 203.0.113.10 - local 198.51.100.5 - ttl 255 - gateway 2001:db8:1234:5678::1 - -{{< /file >}} - - -2. Restart networking services and test the tunnel. Replace `he-ipv6` with the name of your tunnel: - - systemctl restart networking.service - ping6 -I he-ipv6 irc6.oftc.net - - If configuration was successful, you will receive ping replies. If not, check your network configuration for errors. - -### CentOS 7 and Fedora 22+ - -1. Create a file at `/etc/sysconfig/network-scripts/ifcfg-he-ipv6` that contains the following: - - {{< file "/etc/sysconfig/network-scripts/ifcfg-he-ipv6" >}} -NAME="he-ipv6" -DEVICE=he-ipv6 -ONBOOT=yes -USERCTL=yes -BOOTPROTO=none -PEERDNS=no - -IPV6INIT=yes -IPV6_AUTOTUNNEL=yes -IPV6ADDR="2001:db8:1234:5678::2/64" -IPV6_ROUTER=yes -IPV6_AUTOCONF=no - -IPV6_CONTROL_RADVD=yes -IPV6TUNNELIPV4=203.0.113.10 -IPV6TUNNELIPV4LOCAL=45.79.171.199 - -PHYSDEV=eth0 -TYPE=sit -DEVICETYPE=sit -NM_CONTROLLED=no - -IPV6_DEFAULTGW=2001:db8:1234:5678::1 -IPV6_DEFAULTDEV=he-ipv6 - -{{< /file >}} - - - In the above, replace the: - - * `IPV6ADDR` value with your "Client IPV6 Address." - * `IPV6TUNNELIPV4` value with your "Server IPV4 Address." - * `IPV6TUNNELIPV4LOCAL` value with your "Client IPV4 Address." - * `IPV6_DEFAULTGW` value with your "Server IPV6 Address." - -2. Start the `he-ipv6` interface: - - ifup he-ipv6 - -3. Test the tunnel. Replace `he-ipv6` with the name of your tunnel: - - ping6 -I he-ipv6 irc6.oftc.net - - If configuration was successful, you will receive ping replies. If not, check your network configuration for errors. - -### CentOS 6 - -1. Add the following lines to your `/etc/sysconfig/network` file: - - {{< file "/etc/sysconfig/network" >}} -NETWORKING_IPV6=yes -IPV6_DEFAULTDEV=he-ipv6 - -{{< /file >}} - - -2. Create a file at `/etc/sysconfig/network-scripts/ifcfg-he-ipv6` that contains the following: - - {{< file "/etc/sysconfig/network-scripts/ifcfg-he-ipv6" >}} -DEVICE=he-ipv6 -BOOTPROTO=none -ONBOOT=yes -IPV6INIT=yes -IPV6TUNNELIPV4=203.0.113.10 -IPV6ADDR=2001:db8:1234:5678::2/64 - -{{< /file >}} - - - Replace the `IPV6TUNNELIPV4` value with your remote tunnel endpoint and the `IPV6ADDR` value with the "Client IPv6" address provided to you by your tunnel broker. - -3. Start the `he-ipv6` interface: - - ifup he-ipv6 - -4. Test the tunnel. Replace `he-ipv6` with the name of your tunnel: - - ping6 -I he-ipv6 irc6.oftc.net - - If configuration was successful, you will receive ping replies. If not, check your network configuration for errors. - -### Arch Linux - -Refer to the [Arch Linux Wiki](https://wiki.archlinux.org/index.php/IPv6_tunnel_broker_setup) for more information. - -### Gentoo Linux - -Refer to the [Gentoo Linux Wiki](https://wiki.gentoo.org/wiki/IPv6_router_guide) for more information. diff --git a/docs/guides/networking/types-of-networks/index.md b/docs/guides/networking/types-of-networks/index.md deleted file mode 100644 index 968ef866a55..00000000000 --- a/docs/guides/networking/types-of-networks/index.md +++ /dev/null @@ -1,200 +0,0 @@ ---- -slug: types-of-networks -title: "Different Types of Computer Networks, Explained" -description: "What are the different types of networks and what do they do? This guide discusses the main types of computer networks and what they’re used for." -authors: ["David Robert Newman"] -contributors: ["David Robert Newman"] -published: 2023-09-26 -keywords: ['types of networks', 'networking type', 'network connection types', 'what type of network is the internet', 'type of computer networks'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -Networks vary widely in size and scope. Some are simple and highly localized. Others span the globe – and beyond, even going into space. Picking the right network depends on what and where you want to connect. Your type of application also matters in some cases; for example, there are special networks optimized just for storage devices.This guide describes 10 major network types and where each network might, or might not, make sense. - -## Internet - -When you say “the Internet,” you probably mean the massive network that goes everywhere on Earth and beyond, now extending to the International Space Station and even to Mars. But there’s also a generic term "internet" with a lowercase "i", that refers to any network of networks. - -This is a good place to start because the designs used to tie networks together also apply within a single network. Most networks adhere to one of three fundamental designs: Point-to-point, hub-and-spoke, or meshed. - -- **Point-to-point connection**: This represents a direct link between two networks or computers. - -- **Hub-and-spoke design**: This configuration connects multiple end-stations (computers, printers, IoT devices) to a central server; for example, multiple PCs utilizing a single server for file storage. - -- **Meshed networks**: These networks come in two variations, partial and full. In a fully meshed design, all nodes (servers, routers, switches) connect to all other nodes. This provides the greatest amount of redundancy in case of link or node failures, but also the greatest cost if you’re paying for each link. In a partial mesh, only some nodes are interconnected. The global Internet is a partially meshed network. - -The basic building block of the Internet, as well as many other internets, is the TCP/IP protocol suite, a set of open standards that allow all manufacturers’ devices to interoperate. - -**Suitable for**: Connectivity between any set of devices, regardless of manufacturer - -**Not suitable for**: Very old legacy computers that use proprietary communications protocols - -## Personal-Area Network (PAN) - -One of the simplest network types is the PAN, which wirelessly connects a few devices close to you. Devices might include your laptop, phone, tablet, and perhaps earbuds or smart speakers. -Bluetooth is the most common protocol for PAN devices. Without obstacles (people, walls, metal, and so on), Bluetooth has a range of about 10 meters (33 feet). The central concept with PANs is that they’re personal, connecting devices for you and you alone. - -A variant of the PAN is the Car Area Network (CAN), which uses Bluetooth or WLAN (covered below) to link devices within a car. CANs then bridge traffic to a cellular connection to the global Internet. - -**Suitable for**: - -- Tying together personal devices -- Devices in close proximity - -**Not suitable for**: - -- Connecting multiple users’ devices -- Connecting one user’s devices over any significant distance - -## Local-Area Network (LAN) - -When LANs first appeared years ago, they consisted of a few PCs and printers in close proximity, always in an office setting, and always communicating either with a locally attached server or directly with one another. This was a revolutionary departure from previous network designs, where all intelligence lived on a central mainframe or minicomputer, and end-stations were essentially just dumb terminals. - -Today the meaning of LAN is much broader. You may have a wired LAN in your office or at home. You may connect a much broader range of devices to LANs, including surveillance cameras, Internet of Things (IoT) devices, and even solar power controllers in home networks. - -In office settings, it’s common to segment networks into many LANs, with one or more for each workgroup (accounting, engineering, sales, etc.) or each physical area (such as floors in office buildings). - -The key building block for LANs is the Ethernet switch, which ties together devices in each area and provides a common broadcast domain for attached devices. - -Enterprise-grade switches also support virtual LANs (VLANs), which provide multiple segments on the same switch. For example, you might configure two VLANs on a switch, allocating separate groups of ports for users in your accounting and engineering teams. Each VLAN forms its own broadcast domain, meaning neither team can see the other team's traffic. - -**Suitable for**: - -- Wired connection of computers, servers, and printers -- Data sharing among users in a home or office - -**Not suitable for**: - -- Computers across town or the world -- Settings where cable installation isn’t feasible - -## Wireless LAN (WLAN) - -Connections to a wireless LAN work over the air, using radio waves. Most people think "Wi-Fi" when it comes to LAN connections without wires. Wi-Fi is a marketing term coined by an equipment makers’ consortium; the IEEE’s 802.11 working group defines the actual Wi-Fi technical standards. - -In a typical Wi-Fi setup, your laptop, phone, and tablet all connect to a local access point, which then bridges traffic between wireless and wired networks. You can also set up a peer-to-peer WLAN connecting two devices; this is handy in places where there isn’t any other network connectivity (for example, aboard a plane). - -Performance tends to be lower on wireless networks than on wired ones. That’s because the airwaves are a shared-access medium, accessible by only one device at a time. - -In practical terms, this usually means when one station receives or sends traffic, all other connected stations need to remain quiet. There is a Wi-Fi technology called multiple-in, multiple-out (MIMO) where multiple stations can transmit and receive at the same time. MIMO has been implemented on many consumer and enterprise-grade WLAN devices. - -Security can be a concern with WLANs, even though it’s possible to secure them at least as well as wired networks. In general, you never want to connect to an open access point at public places such as cafes or airports without also using a virtual private network (VPN, covered below). - -**Suitable for**: - -- Over-the-air connectivity for computers, phones, tablets, and printers -- Settings where cable installation isn’t feasible - -**Not suitable for**: - -- Computers across town or the world -- Applications where high performance is critical - -## Virtual Private Network (VPN) - -Most networks carry traffic using the TCP/IP protocol suite – but as originally designed, TCP/IP does nothing to secure traffic against spoofing or interception. VPNs add security by creating a private network overlay atop any public network. - -There are many VPN types, but all provide the key functions of authentication and encryption. Authentication simply means each side verifies its identity, typically using certificates or other digital keys. This is an important first step since you don’t want just anyone connecting to your secure site. Encryption is the means of encoding and decoding data in a way that’s unreadable by attackers. - -VPNs use authentication to set up a secure tunnel across a public network, and then send encrypted traffic through that tunnel. The tunnel endpoints could be anywhere: You could set up a site-to-site VPN, where you connect two office locations, or a road-warrior VPN, where one endpoint is your laptop and the other is your office location or a server in your cloud deployment. One potential drawback is that VPNs add overhead, degrading performance. - -**Suitable for**: - -- Protection of data in flight, even across unsecured networks -- Remote access from hotels, airports, and cafes -- Remote access to sensitive internal networks and servers - -**Not suitable for**: - -- Applications where maximum performance is important - -## Storage Area Network (SAN) - -As your server count grows, it’s often more efficient to provide pools of shared storage rather than equipping each new server with extensive local storage. That’s where SANs come in. They provide a dedicated high-speed network connecting servers with shared storage pools. - -SANs differ from network-attached storage (NAS) devices, where all disks reside within a single chassis. In contrast, SANs allow shared storage pools made up of many storage arrays. SANs also use specialized storage protocols such as Fibre Channel or iSCSI optimized to speed input/output. Networks running Fibre Channel often use SAN switches to tie together servers and storage pools. - -**Suitable for**: - -- High-speed access to shared storage pools -- Organizations in need of highly scalable data storage - -**Not suitable for**: - -- General-purpose network connectivity - -## Passive Optical Network (PON) - -Although most LANs use copper cabling, fiber-optic cabling not only has far higher bandwidth capacity but also can carry different users' traffic over much longer distances. Fiber's unique characteristics enable PONs, which use optical splitters to divide and combine user traffic. Telecom providers and ISPs typically provision PONs. - -In a PON design, an optical line terminal (OLT) serves as a central concentrator, connecting the network to the rest of the world. Optical splitters connected to an OLT then divide and combine user traffic respectively headed downstream and upstream. At the network edge, an optical network termination (ONT) device terminates the upstream fiber connection and bridges traffic to and from local devices, such as computers and phones, connected via copper patch cables. - -The "passive" part of PON indicates that optical components don’t require power to drive signals, reducing power and heat. Another potential energy saving: The single-mode fiber cabling in PONs can carry signals much further than copper cabling, eliminating the need for powered repeaters. Depending on the transceivers used, a single run of single-mode fiber can carry traffic up to 40 kilometers. In contrast, the distance limit for the most common copper cabling is 100 meters. - -Single-mode fiber tends to be more expensive than copper, mainly due to transceiver and fiber termination costs. While PONs can be an excellent option in new deployments, you need to factor in the cost of ripping out copper cabling when evaluating them as a replacement technology. - -**Suitable for**: - -- Connectivity over relatively long distances -- Networks with limited power and heat budgets - -**Not suitable for**: - -- Networks with significant investment in existing cabling - -## Metropolitan-Area Network (MAN) - -MANs connect networks and computers within a wider range than any LAN could. A single MAN could encompass one city, multiple cities and towns, or a large area within a city with multiple large buildings. - -MANs usually employ fiber-optic cabling to connect LANs. It’s a common practice for telecom providers and ISPs to provision “dark fiber” (unused capacity) around a city, and then “light” the fiber as they connect customer networks. Sometimes cities build their own dark fiber networks, and then lease capacity to telecom providers, ISPs, and private companies. Metro Ethernet is an interesting connectivity choice; in recent years, the LAN technology has been extended to connect systems over distances as great as 100 km (about 62 miles). - -MANs are a good option when you have multiple sites within a metropolitan region. You may be able to create a virtual overlay network using existing dark fiber. - -**Suitable for**: - -- Connectivity within a metropolitan area or large campus -- Connectivity that can be provisioned and managed by an ISP or telecom provider - -**Not suitable for**: - -- Smaller LANs or campuses - -## Wide-area network (WAN) - -While the global Internet is the best-known example of a WAN, the term generically refers to any network that connects devices or networks over long distances. Telecom providers and ISPs operate WANs and sell capacity to national and multinational customers to link various business sites. - -Some large enterprises operate their own WANs, leasing private lines from telecom providers or even running their own cabling to directly connect locations. While private WANs were more common before the rise of the global Internet, they are still in use. Large organizations may want to avoid putting sensitive traffic on public networks, or may lease private lines to ensure performance is always predictable. - -**Suitable for**: - -- Connectivity across far-flung locations -- Connectivity of arbitrarily large numbers of devices - -**Not suitable for**: - -- Designs where all devices are in close proximity - -## Software-Defined WAN (SD-WAN) - -While many WANs carry TCP/IP traffic, they often aren’t TCP/IP networks themselves. Instead, WANs usually move traffic using lower-layer technologies such as multiprotocol label switching (MPLS), long-term evolution (LTE), and an alphabet soup of older link-layer technologies. WANs running TCP/IP can be difficult to configure and manage. - -The variety and complexity of WAN technologies gave rise to software-defined networking, which treats the entire WAN as an abstraction. With SD-WAN, you don’t worry about the physical network; instead, you use centralized management software to set up and manage all flows. - -This is a radical departure from decades of WAN management practice. Previously, if you designed a WAN using, say, a mix of MPLS and metropolitan Ethernet circuits, you’d have to think about how you’d interconnect the technologies, what type of devices resided within each network, and how you’d configure them. - -SD-WAN software hides all that complexity. You configure the management software to move traffic between points without worrying about what’s between those points. SD-WANs automatically and continuously learn network topology, adapting in real-time to changes in the underlying network and routing traffic around outages. - -Initial setup with SD-WANs is also much faster. Provisioning and configuring WANs often took weeks or months; SD-WAN can reduce that time to minutes, greatly improving agility. - -**Suitable for**: - -- Complex WAN designs -- New WAN overlays - -**Not suitable for**: - -- Simple LAN or MAN installations - -## Conclusion - -Since these network types greatly differ in terms of scope and technology, it's important to accurately assess which type makes sense for you. There are really just two questions: What kinds of devices are you looking to connect, and where are they? Armed with this networking knowledge, you can make an informed decision about which network types make the most sense for you. diff --git a/docs/guides/networking/vpn/set-up-wireguard-vpn-on-debian/index.md b/docs/guides/networking/vpn/set-up-wireguard-vpn-on-debian/index.md deleted file mode 100644 index 6e285e91186..00000000000 --- a/docs/guides/networking/vpn/set-up-wireguard-vpn-on-debian/index.md +++ /dev/null @@ -1,290 +0,0 @@ ---- -slug: set-up-wireguard-vpn-on-debian -title: "Set Up WireGuard VPN on Debian" -description: 'WireGuard encrypts your traffic quickly and safely. This guide will show you how to set up a Wireguard VPN server and client on Debian.' -og_description: 'This guide will show you how to install WireGuard, a fast and secure VPN, on Linode.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2019-07-08 -modified: 2021-10-15 -keywords: ['wireguard','vpn','debian'] -tags: ["networking","security","vpn","debian"] -bundles: ['network-security'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -relations: - platform: - key: setup-wireguard-vpn - keywords: - - distribution: Debian 9 -aliases: [] ---- - -## What is WireGuard? - -[WireGuard](https://www.wireguard.com)® is a simple, fast, and secure VPN that utilizes state-of-the-art cryptography. With a small source code footprint, it aims to be faster and leaner than other VPN protocols such as [OpenVPN](https://en.wikipedia.org/wiki/OpenVPN) and [IPSec](https://en.wikipedia.org/wiki/IPsec). WireGuard is still under development, but even in its unoptimized state it is faster than the popular OpenVPN protocol. - -WireGuard sets up standard network interfaces (such as `wg0` and `wg1`), which behave much like the commonly found `eth0` interface. This makes it possible to configure and manage WireGuard interfaces using standard tools such as [`ifconfig`](https://en.wikipedia.org/wiki/Ifconfig) and `ip`. Currently, WireGuard is only available on Linux. - -Configuring WireGuard is as simple as setting up SSH. A connection is established by an exchange of public keys between server and client. Only a client that has its public key in its corresponding server configuration file is allowed to connect. A WireGuard server's configuration file resembles the following example: - - {{< file "/etc/wireguard/wg0.conf" conf >}} -[Interface] -PrivateKey = -Address = 10.0.0.1/24, fd86:ea04:1115::1/64 -ListenPort = 51820 -PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE -SaveConfig = true - -[Peer] -PublicKey = -AllowedIPs = 10.0.0.2/24, fd86:ea04:1115::0/64 - {{< /file >}} - -In this guide you will learn how to: - -* [Configure a WireGuard server](#configure-wireguard-server) on a Linode running Debian 9. -* [Configure a WireGuard client](#configure-wireguard-client) on your local computer or another Linode. -* [Establish a simple peer connection](#connect-the-client-and-server) between your WireGuard server and client. -{{< note type="alert" >}} -Do not use WireGuard for critical applications. The project is still undergoing security testing and is likely to receive frequent major updates in the future. -{{< /note >}} - -## Before You Begin - -- [Deploy a Linode](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) running Debian 9. -- [Add a limited user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account) with `sudo` privileges to your Linode. -- Set your system's [hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname). - -{{< note >}} -The `GRUB 2` kernel is required for this guide. All distributions for all new Linodes now boot with the `GRUB 2` kernel by default. However, if you are running an older distribution, you will need to check to see which kernel you are running. You can use the [Update Kernel Guide](https://techdocs.akamai.com/cloud-computing/docs/manage-the-kernel-on-a-compute-instance) to check your kernel version and change it using the Cloud Manager. Select `GRUB 2` from the *Boot Settings: Select a Kernel* dropdown menu in Step 4 of [Update Your Linode Kernel with Linode's Cloud Manager](https://techdocs.akamai.com/cloud-computing/docs/manage-the-kernel-on-a-compute-instance#view-and-modify-the-kernel-in-cloud-manager). -{{< /note >}} - -## Install WireGuard - -1. Add the WireGuard repository to your sources list. Apt will automatically update the package cache. - - echo "deb http://deb.debian.org/debian/ unstable main" > /etc/apt/sources.list.d/unstable-wireguard.list - printf 'Package: *\nPin: release a=unstable\nPin-Priority: 150\n' > /etc/apt/preferences.d/limit-unstable - -1. Update your packages and install WireGuard and WireGuard tools. DKMS (Dynamic Kernel Module Support) will build the WireGuard kernel module. - - apt update - apt install wireguard-dkms wireguard-tools - - If successful, you'll see the following output: - - {{< output >}} -wireguard: -Running module version sanity check. - - Original module - - No original module exists within this kernel - - Installation - - Installing to /lib/modules/4.9.0-9-amd64/updates/dkms/ - -depmod... - -DKMS: install completed. -Processing triggers for libc-bin (2.24-11+deb9u4) ... -{{< /output >}} - -## Configure WireGuard Server - -1. Navigate to the `/etc/wireguard` directory and generate a private and public key pair for the WireGuard server: - - sudo umask 077 - sudo wg genkey | tee privatekey | wg pubkey > publickey - - This will save both the private and public keys; they can be viewed with `cat privatekey` and `cat publickey` respectively. - -1. Create the file `/etc/wireguard/wg0.conf` and add the contents indicated below. You'll need to enter your server's private key in the `PrivateKey` field, and its private IP addresses in the `Address` field. Refer to the list below the example for more details. - - {{< file "/etc/wireguard/wg0.conf" conf >}} -[Interface] -PrivateKey = -Address = 10.0.0.1/24, fd86:ea04:1115::1/64 -ListenPort = 51820 -PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE -SaveConfig = true -{{< /file >}} - - - **PrivateKey** the server's private key generated in above. - - - **Address** defines the private IPv4 and IPv6 addresses for the WireGuard server. Each peer in the VPN network should have a unique value for this field. Typical values are `10.0.0.1/24`, `192.168.1.1/24`, or `192.168.2.1/24`. This is not the same as a private IP address that Linode can assign to your Linode instance. - - - **ListenPort** specifies which port WireGuard will use for incoming connections. The default is `51820`. What you set here you will need to reference in your firewall settings later. - - - **PostUp** and **PostDown** defines steps to be run after the interface is turned on or off, respectively. In this case, `iptables` is used to set Linux IP masquerade rules to allow all the clients to share the server's IPv4 and IPv6 address. The rules will then be cleared once the tunnel is down. - - - **SaveConfig** tells the configuration file to automatically update whenever a new peer is added while the service is running. - -### Set Up Firewall Rules - -1. Install UFW: - - sudo apt-get install ufw - -1. Allow SSH connections and WireGuard's VPN port: - - sudo ufw allow 22/tcp - sudo ufw allow 51820/udp - sudo ufw enable - -1. Verify the settings: - - sudo ufw status verbose - - -### Start the WireGuard Service - -1. Start WireGuard: - - sudo wg-quick up wg0 - - {{< note respectIndent=false >}} -`wg-quick` is a convenient wrapper for many of the common functions in `wg`. You can turn off the wg0 interface with `wg-quick down wg0` -{{< /note >}} - -1. Enable the WireGuard service to automatically restart on boot: - - sudo systemctl enable wg-quick@wg0 - -1. Check if the VPN tunnel is running with the following two commands: - - sudo wg show - - You should see a similar output: - - {{< output >}} -user@debian:/# wg show -interface: wg0 - public key: Nrl2nVQxSwrKrvz6jQcrsziuVRPWT9N1Q8/yaQkAXUg= - private key: (hidden) - listening port: 51820 -{{< /output >}} - - You may need to install [net-tools](https://tracker.debian.org/pkg/net-tools) to run `ifconfig`. Use `sudo apt-get install net-tools` if needed. - - sudo ifconfig wg0 - - Your output should resemble the following: - - {{< output >}} -user@debian:/# ifconfig wg0 -wg0: flags=209 mtu 1420 - inet 10.0.0.1 netmask 255.255.255.0 destination 10.0.0.1 - inet6 fd86:ea04:1115::1 prefixlen 64 scopeid 0x0 - unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 1 (UNSPEC) - RX packets 0 bytes 0 (0.0 B) - RX errors 0 dropped 0 overruns 0 frame 0 - TX packets 0 bytes 0 (0.0 B) - TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 -{{< /output >}} - - -## Configure WireGuard Client - -The process for setting up a client is similar to setting up the WireGuard server. When using Debian as your client's operating system, the only difference between the client and the server is the configuration file. In this section, you will configure a WireGuard client on Debian 9. - -{{< note >}} -For installation instructions on other operating systems, see the [WireGuard docs](https://www.wireguard.com/install/). -{{< /note >}} - -1. Follow the steps in the [Install WireGuard](#install-wireguard) section of the guide. - -1. Once you have installed WireGuard, follow the steps in the [Configure WireGuard Server](#configure-wireguard-server) section. Replace the example configuration file with the example file below. - - {{< file "/etc/wireguard/wg0.conf" conf >}} -[Interface] -PrivateKey = -Address = 10.0.0.2/24, fd86:ea04:1115::5/64 - {{< /file >}} - - The difference between the client and the server's configuration file, `wg0.conf`, is it contains **its own** IP addresses and does not contain the `ListenPort`, `PostUP`, `PostDown`, or `SaveConfig` values. - - -1. [Set up Firewall rules](#set-up-firewall-rules) on your WireGuard client. - -1. [Start the WireGuard Service](#start-the-wireguard-service). - -## Connect the Client and Server - -1. Stop the interface with `sudo wg-quick down wg0` on both the client and the server. - -1. Edit the `wg0.conf` file on the client to add the server's public key, public IP address, port, and allowed IPs. - - {{< file "/etc/wireguard/wg0.conf" conf >}} -[Peer] -PublicKey = -Endpoint = :51820 -AllowedIPs = 10.0.0.1/24, fd86:ea04:1115::1/64 -{{< /file >}} - -1. Edit the `wg0.conf` file on the server to add the client's public key and allowed IPs. - - {{< file "/etc/wireguard/wg0.conf" conf >}} -[Peer] -PublicKey = -AllowedIPs = 10.0.0.2/24, fd86:ea04:1115::5/64 -{{< /file >}} - -1. Restart the `wg` service on both the server and the client: - - sudo wg-quick up wg0 - -1. You can also add peers to the server from the command line. This information will be added to the config file automatically because of the `SaveConfig` option specified in the `wg0.conf` file. - - Run the following command from the server. Replace the example IP addresses with those of the client: - - sudo wg set wg0 peer allowed-ips 10.0.0.2/24,fd86:ea04:1115::5/64 - -1. Verify the connection. The following command can be run from both the client or the server: - - sudo wg - - Regardless of which method you choose to add peer information to WireGuard, the **Peer** section appears in the output of the `sudo wg` command if the setup was successful. - - {{< output >}} -user@debian:/# sudo wg -interface: wg0 - public key: vD2blmqeKsV0OU0GCsGk7NmVth/+FLhLD1xdMX5Yu0I= - private key: (hidden) - listening port: 51820 - -peer: iMT0RTu77sDVrX4RbXUgUBjaOqVeLYuQhwDSU+UI3G4= - endpoint: 10.0.0.2:51820 - allowed ips: 10.0.0.2/24, fd86:ea04:1115::/64 -{{< /output >}} - - This Peer section will be automatically added to `wg0.conf` when the service is restarted. If you would like to add this information immediately to the config file, you can run: - - sudo wg-quick save wg0 - - Additional clients can be added using the same procedure. - -### Test the Connection - -1. Return to the client and ping the server: - - ping 10.0.0.1 - -1. Once you've successfully established the ability to ping the server from the client, run the following command: - - sudo wg - - The last two lines of the output from running the `wg` command should be similar to: - - {{< output >}} - latest handshake: 1 minute, 17 seconds ago - transfer: 98.86 KiB received, 43.08 KiB sent - {{}} - - This indicates that you now have a private connection between the server and client. If you did not successfully ping the server from the client you will not see these lines. You can also ping the client from the server to verify that the connection works both ways. - - -## Next steps - -The process used in this guide can be extended to configure network topologies. As mentioned previously, WireGuard is an evolving technology. If you use WireGuard, you should monitor the [official documentation](https://www.wireguard.com/) and [todo list](https://www.wireguard.com/todo/) for critical updates and new/upcoming features. - -"WireGuard" is a registered trademark of Jason A. Donenfeld. diff --git a/docs/guides/platform/migrate-to-linode/migrate-a-lamp-website-to-linode/index.md b/docs/guides/platform/migrate-to-linode/migrate-a-lamp-website-to-linode/index.md deleted file mode 100644 index 3d7e212f05e..00000000000 --- a/docs/guides/platform/migrate-to-linode/migrate-a-lamp-website-to-linode/index.md +++ /dev/null @@ -1,145 +0,0 @@ ---- -slug: migrate-a-lamp-website-to-linode -title: How to Migrate a LAMP Website to Linode -description: 'How to migrate a LAMP website from another hosting provider to Linode.' -authors: ["Nathan Melehan"] -contributors: ["Nathan Melehan"] -published: 2018-07-31 -keywords: ["lamp", "migrate", "website migration"] -tags: ["linode platform","lamp"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- - -This guide describes how to migrate a website running in a [LAMP](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-18-04#what-is-a-lamp-stack) environment on another host to a new Linode. Read the [Best Practices when Migrating to Linode](/cloud/guides/best-practices-when-migrating-to-linode) guide prior to following this guide for more information about migrating your site. - -This guide includes commands that need to be run at the command line of your current host, which may not be available if you have a shared hosting environment. Ubuntu 18.04 is used as the distribution for the new Linode deployment in this guide. If you'd like to choose another distribution, us the examples here as an approximation for the commands you'll need to run. - -## Migrate Your System - -### Deploy Your Linode - -1. Follow Linode's [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guide and choose Ubuntu 18.04 as your Linux image when deploying. Choose a Linode plan with enough storage space to accommodate the website data from your current host. - -1. Follow the [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide and create a limited Linux user with `sudo` privileges. The examples below assume this user is named `linode_user`. - -### Install LAMP - -1. [Connect to your Linode via SSH.](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#connect-to-the-instance) - -1. If you did not do so previously, update your software: - - sudo apt update && sudo apt upgrade - -1. Install and use `tasksel` to install the `lamp-server` metapackage. - - sudo apt install tasksel - sudo tasksel install lamp-server - -### Prepare and Back up Your Current Host - -The data that needs to be transferred includes: - -- Files from the website document root. This guide assumes this to be `/var/www/html/`, but your server may have it located in another directory. -- MySQL database data (which will be exported using the `mysqldump` utility) -- Apache's configuration files. Specifically, `/etc/apache2/apache2.conf` and `/etc/apache2/sites-available/example.com.conf`. - - These and the files in the `sites-available` directory may have different names. - -Your server may have relevant website data stored in other directories, but these are the common locations for most files in a LAMP deployment. - -Perform a *database dump* needs on your MySQL process prior to transferring the data. This will result in a file on disk that encapsulates your database data which can then be copied over the network as a normal file. -{{< note type="alert" >}} -Stopping services on your current host will temporarily disable your website. -{{< /note >}} - -1. Connect to your current host via SSH. - -1. Stop the web server on your current host: - - sudo systemctl stop apache2 - - This prevents new requests on your website, which will stop new writes to your database. This ensures that the MySQL backup that is taken does not result in an inconsistent dataset. - -1. Perform a database dump of the entire database on your current host: - - sudo mysqldump --all-databases --single-transaction --quick --lock-tables=false > full-backup-$(date +%F).sql -u root -p - - This command results in a file called `full-backup-*.sql` with the current date inserted. - -1. (Optional) Restart your web server on the current host to continue serving visitors in the interim: - - sudo systemctl start apache2 - - {{< note respectIndent=false >}} -If any new information is added to the database on the current host prior to fully transferring service to Linode, that new information is not included in the MySQL backup that was performed in this section. -{{< /note >}} - -### Transfer Data to Your Linode - -The following commands copy files into the home directory of your Linode's Linux user, `~/document_root` in these examples. The section after ensures that these files are moved to the right locations on disk and that their file ownership is correct. This work is done in two steps because the Linux user is assumed to have limited privileges and is unable to write to certain locations on disk. - -1. From your current host, upload the Apache configuration files to your new Linode: - - rsync -az /etc/apache2/apache2.conf linode_user@linode_ip_address:/etc/apache2/ - rsync -az /etc/apache2/sites-available/example.com.conf linode_user@linode_ip_address:~ - -1. Upload the website files to your Linode: - - rsync -av /var/www/html/ linode_user@linode_ip_address:~/document_root - -1. Upload the database dump file to your Linode: - - rsync -az full-backup-*.sql linode_user@linode_ip_address:~ - -### Finish Setting up Your Linode - -1. From your Linode, move the transferred Apache configuration files to the appropriate locations: - - cd ~ - sudo mv apache2.conf /etc/apache2 - sudo mv example.com.conf /etc/apache2/sites-available - -1. Set `root` as the owner and group for the files: - - sudo chown root:root /etc/apache2/apache2.conf /etc/apache2/sites-available/example.com.conf - -1. Move the website files and set the owner and group to `www-data`: - - sudo mv document_root/* /var/www/html - sudo chown -R www-data:www-data /var/www/html - {{< note type="alert" respectIndent=false >}} -This will overwrite all current data in the MySQL database system of your Linode. It is not recommended that you perform this command on a Linode with other existing websites. -{{< /note >}} - -1. Restore the database dump file. Replace `full-backup-*.sql` with the name of your file: - - sudo mysql -u root < full-backup-*.sql - -1. [Reload MySQL's grant tables](https://dev.mysql.com/doc/refman/8.0/en/privilege-changes.html). This is necessary when a web application includes a user in the `mysql.users` table. If this is not performed, the application will not have permission to read from the database: - - sudo mysqladmin flush-privileges -u root -p - -1. Disable the default Apache example site and enable yours: - - sudo a2dissite 000-default.conf - sudo ensite example.com.conf - -1. Restart Apache: - - sudo systemctl restart apache - -### Test Your New Environment - -Go to your Linode's IP address in a web browser. Your website should appear. - -If your website does not load normally, one possible reason is that your IP address could be hard-coded in some areas of the website files or in the database. If this is the case, consult your PHP application framework's documentation for ways to search for and replace those values. For example, WordPress's [WP-CLI interface](https://developer.wordpress.org/cli/commands/search-replace/) and Drupal's [Drush interface](https://www.drupal.org/project/sar) provide methods that help with this task. - -Another reason the site may not load is if your website configuration expects your domain name to be supplied in the HTTP headers of a web request. When you visit your IP directly, this information is not supplied in your request. The [Previewing Websites Without DNS](/cloud/guides/previewing-websites-without-dns) guide describes a workaround for this issue. When you have updated your DNS records, the workaround will no longer be necessary to view your site. - -If you are seeing any other errors on your site, try reviewing Apache's error logs for further clues. The locations for these logs will be listed in your `/etc/apache2/apache2.conf` or `/etc/apache2/sites-available/` files. - -## Migrating DNS Records - -The last step required to migrate is to update your DNS records to reflect your new Linode's IP. Once this is done, visitors will start loading the page from your Linode. - -{{% content "use-linode-name-servers" %}} diff --git a/docs/guides/quick-answers/linux-essentials/multi-cloud-vs-hybrid-cloud/index.md b/docs/guides/quick-answers/linux-essentials/multi-cloud-vs-hybrid-cloud/index.md deleted file mode 100644 index 9edc86738bb..00000000000 --- a/docs/guides/quick-answers/linux-essentials/multi-cloud-vs-hybrid-cloud/index.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -slug: multi-cloud-vs-hybrid-cloud -title: "Multicloud vs. Hybrid Cloud: Differences Explained" -title_meta: "What is the Difference Between Hybrid Cloud and Multicloud?" -description: 'Comparing multicloud vs hybrid cloud? Discover their similarities, differences, and the pros and cons of each approach. Find the right one for you by reading this guide.' -authors: ["Jack Wallen"] -contributors: ["Jack Wallen"] -published: 2022-01-28 -keywords: ['hybrid cloud vs multi cloud','hybrid vs multi cloud','multi-cloud vs hybrid cloud'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -In the last couple of years, *the cloud* has become a ubiquitous term and piece of technology. But depending on who you ask, what is meant by "the cloud" can vary dramatically. For some, the cloud is only a way to store files and photos on a third-party service. For others, the cloud is a means to sync data between phones and desktops. Others look at the cloud as a method of serving applications to consumers and customers. As you keep climbing up the IT ladder, the definition gets more granular and more complicated. - -Why? Because the technology driving the cloud becomes more complicated. - -For businesses, the cloud is a very important piece of the puzzle. One that makes it possible for companies to better monetize their offerings, save money, and deliver reliable services that scale to meet demand. It's on this level that things become more complex. Part of the reason for this confusion is two different types of cloud technologies that are often mistaken with one another. - -Hybrid vs. multicloud environments are quite pervasive in the IT landscape. There are differences between hybrid and multicloud deployments that are important to understand when a company designs its cloud strategy. - -## What is Multicloud Computing? - -Multicloud deployment is often preferred by businesses, sometimes without the business recognizing it is adopting the technology, but doing so because it addresses the business requirements. It's association by need, not design. - -The defining aspect of the multicloud setup is that it makes use of multiple public cloud services. These services often come from different providers, all of which make their offerings publicly available. So your company might use any combination of AWS, Google Cloud, Azure, Linode, Digital Ocean, or RackSpace. For example, your groupware might be hosted on Google Cloud (such as Google Workspaces), your mobile applications on AWS, your containerized applications on Linode, your website on Azure, and your storage on Digital Ocean. That's a widespread multicloud deployment, but it illustrates how multicloud can work. - -One reason to spread your technology net so wide is because one cloud provider doesn't support all of the technologies your business requires to function. Another reason could be that it’s more cost-effective to run a particular type of workload on one platform than the others. - -IT decision makers align company business technology adoption with a need and solve that need with a solution. In the modern IT landscape, that solution is often handled via the cloud. Given how many pre-built services exist within the cloud, the decision is not difficult. *Problem X + Cloud Solution Y = Reliable Workflow*. - -## What is Hybrid Cloud Computing? - -Hybrid cloud computing makes use of a private cloud in addition to public cloud hosting platforms. In this environment, both the public and the private cloud environments are managed as one. - -Hybrid cloud computing is more complicated than multicloud for two reasons: - -- You must include a private cloud option. -- You must be able to seamlessly combine both private and public options. - -For example, you have a service that you run on-premise. It's a containerized application, run via a private cloud, and your container database is served up via your in-house data center. That application runs well during non-peak periods. But during peak demand, your private cloud isn't capable of scaling to meet the higher demand. - -To make that work, you employ frameworks and services that automatically shift the load from your in-house private cloud to your public cloud to handle the added demand. Once peak times end, the load shifts back to your private cloud. - -As far as your customer is concerned, nothing changes. For your business, considerable effort goes into making that a reality because there are a lot of moving pieces required for this to work smoothly. When this type of deployment functions as expected, it's a game-changer. Service is seamless and the technology making it happen is transparent. - -## Multicloud vs. Hybrid Cloud - -These two technologies are similar, but their differences matter when planning your cloud strategy. - -### Similarities Between Hybrid Cloud and Multicloud - -The obvious similarity in hybrid cloud vs multicloud is that they both make use of public cloud technology. Without public cloud technology, neither of these types would exist. Because they both use public clouds, these two types of deployments co-exist well. For example, your business can make use of a multicloud deployment for one department, or location, and a hybrid cloud for another. Those two deployments are able to communicate with one another. - -A hybrid cloud can become a multicloud deployment if the hybrid cloud uses multiple public clouds and multiple private clouds combined. - -### Difference Between Hybrid Cloud and Multicloud - -The biggest difference between the multicloud and hybrid cloud is that in multicloud only public clouds are used and in the hybrid cloud, a mixture of public and private clouds are used. It's that private cloud that makes the difference. The difference is more telling than you might think. - -Any business can quickly get up and running with a multicloud environment with little effort because a third party takes care of almost everything. For example, with Google Workspaces, the only thing your IT staff does is add branding, usernames, and policies. - -Consider if you opt to pull a particular functionality in-house with a private instance of Nextcloud. Now your IT department has to deploy the platform, configure it to function within your business, and then add branding, users, etc. After that, you have to connect that private cloud so it functions with your public cloud tools. Hybrid cloud environments are just more complex. - -It’s important to clarify one thing: A private cloud doesn't have to be hosted in-house. Most third-party cloud services can provide you with a private cloud option. The difference here is that although the public has access to the third-party provider, they do not have access to the private cloud that the service provides you. - -### Pros and Cons of Hybrid Cloud and Multicloud - -Multicloud is considerably easier to deploy and use. Typically once your multicloud environment is up and running, it is very reliable because those third-party cloud vendors have a vested interest in keeping your business functioning. That equates to your IT staff being able to spend their precious time doing more important things. - -When comparing multicloud vs hybrid cloud, there's the issue of cost. This is where the concept of *Cloud Bursting* comes into play. This is when you have applications and services that normally run on your private cloud but, when demand increases, the applications/services automatically "burst" out to your public cloud space to meet demand. There are three types of cloud bursting: distributed load balancing (used to simultaneously provision cloud resources), manual bursting (used to manually provision and deprovision cloud resources and services), and automated bursting (used to automatically provision and deprovision cloud resources and services). This saves you considerable money, increases the efficiency of operations, and improves performance and productivity of your deployments. Cloud bursting is a viable solution for applications that must read data from storage, database applications that require sharding for higher performance, applications that depend on massive amounts of data, and AL/ML models. However, if your applications rely on low-latency or you have simulations that create large amounts of node-to-node traffic, cloud bursting might not be a great fit. Other issues to factor in with cloud bursting are the complications involved in setting up the services, and keeping them running smoothly. You need to implement observability services and tools to keep close watch. - -Private clouds, especially those you run either in-house or on a third-party host, are significantly cheaper than their public counterparts. Think about it this way: You can pay for each employee to use Google Workspaces, or you can deploy Nextcloud in-house and use it for free. You are only paying the premium price when it's absolutely necessary. - -The downfall of such a setup is that it's complicated to deploy. You need an IT staff with considerable skills to get the hybrid cloud up and running and maintain it. - -Another benefit of the hybrid cloud is that it allows your business to keep sensitive information in-house. You might have client information or company IP that you don't want to find its way into the public. For such a case, the hybrid cloud is the best option, because you keep control over what happens with that data. - -Control is another benefit of the hybrid cloud. With a multicloud environment, you are only able to control so much of the platform because your third-party provider is only willing to hand over so much configuration to your company and staff. With the hybrid cloud, at least on the private cloud end, you are in full control. This is especially true when using open-source solutions on your private cloud, where your in-house developers can dive into the code and make changes to perfectly suit your company. - -## Hybrid Cloud vs Multicloud: Which Model Is Right For Me? - -You can boil this decision down by answering a simple question. Decide whether cost savings, simplicity, or control is most important to your company. If your business is looking to keep costs down, while keeping control of your data and customizations, then a hybrid cloud is probably the best option. However, if your business just needs everything up and running, without worrying about downtime, maintenance, or staffing, the multicloud environment might be the perfect solution. Keep in mind that in the long run the multicloud environment will probably cost you more and won't offer you nearly the control you have with a hybrid cloud. - -## Conclusion - -The type of cloud strategy that you use is a decision to make upfront. It's important to understand that you can, at any time, switch routes. If you already have a multicloud environment, you can layer on a private cloud and turn the multi to a hybrid cloud deployment. Cloud host providers like Linode make adopting either as simple as signing up for your account, deploying your services, and then bridging those together with your other cloud deployments. - -There's plenty of [documentation](/cloud) to help you make this a reality. - -If you find the multicloud costs too much, or that it does not give you the control or security your company demands, you're just a private cloud deployment away from solving that problem. diff --git a/docs/guides/quick-answers/linux/delete-file-linux-command-line/index.md b/docs/guides/quick-answers/linux/delete-file-linux-command-line/index.md deleted file mode 100644 index 5036821d484..00000000000 --- a/docs/guides/quick-answers/linux/delete-file-linux-command-line/index.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -slug: delete-file-linux-command-line -title: "Use rm to Delete Files and Directories on Linux" -description: "This guide provides you instructions for deleting files, directories, content and more using rm command from the Linux Command Line on any distribution." -authors: ["Edward Angert"] -contributors: ["Edward Angert"] -published: 2018-07-03 -modified: 2022-01-14 -keywords: ["remove files", "delete files", "rm"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -tags: ["linux"] -aliases: [] ---- - -This guide shows how to use `rm` to remove files, directories, and other content from the command line in Linux. - -{{< note >}} -To avoid creating examples that might remove important files, this Quick Answer uses variations of `filename.txt`. Adjust each command as needed. -{{< /note >}} - -## The Basics of Using rm to Delete a File - -- Delete a single file using `rm`: - - rm filename.txt - -- Delete multiple files: - - rm filename1.txt filename2.txt - -- Delete all `.txt` files in the directory: - - rm *.txt - -## Options Available for rm - -### `-i` Interactive mode - -Confirm each file before delete: - - rm -i filename.txt - -### `-f` Force - -Remove without prompting: - - rm -f filename.txt - -### `-v` Verbose - -Show report of each file removed: - - rm -v filename*.txt - -### `-d` Directory - -Remove the directory: - - rm -d filenames/ - -**Note:** This option only works if the directory is empty. To remove non-empty directories and the files within them, use the `r` flag. - -### `-r` Recursive - -Remove a directory and any contents within it: - - rm -r filenames/ - -### Combine Options - -Options can be combined. For example, to remove all `.png` files with a prompt before each deletion and a report following each: - - rm -iv *.png - -{{< output >}} -remove filename01.png? y -filename01.png -remove filename02.png? y -filename02.png -remove filename03.png? y -filename03.png -remove filename04.png? y -filename04.png -remove filename05.png? y -filename05.png -{{< /output >}} - -### `-rf` Remove Files and Directories, Even if Not Empty - -Add the `f` flag to a recursive `rm` command to skip all confirmation prompts: - - rm -rf filenames/ - -## Combine rm with Other Commands - -### Remove Old Files Using find and rm - -Combine the [find command](/cloud/guides/find-files-in-linux-using-the-command-line)'s `-exec` option with `rm` to find and remove all files older than 28 days old. The files that match are printed on the screen (`-print`): - - find filename* -type f -mtime +28 -exec rm '{}' ';' -print - -In this command's syntax, `{}` is replaced by the `find` command with all files that it finds, and `;` tells `find` that the command sequence invoked with the `-exec` option has ended. In particular, `-print` is an option for `find`, not the executed `rm`. `{}` and `;` are both surrounded with single quote marks to protect them from interpretation by the shell. \ No newline at end of file diff --git a/docs/guides/quick-answers/linux/drupal-with-docker-compose/index.md b/docs/guides/quick-answers/linux/drupal-with-docker-compose/index.md deleted file mode 100644 index dd55de2f7de..00000000000 --- a/docs/guides/quick-answers/linux/drupal-with-docker-compose/index.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -slug: drupal-with-docker-compose -title: Install Drupal with Docker Compose -description: 'This guide provides you with step-by-step instructions for installing and configuring Drupal, a popular content management system, using Docker Compose.' -authors: ["Nathan Melehan"] -contributors: ["Nathan Melehan"] -published: 2018-06-29 -keywords: ["Docker", "Docker Compose", "Drupal"] -tags: ["postresql","docker","drupal","linux"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Official Drupal Image on Docker Hub](https://hub.docker.com/_/drupal/)' - - '[Overview of Docker Compose | Docker Documentation](https://docs.docker.com/compose/overview/)' -aliases: [] ---- - -## What Are Docker and Docker Compose? - -**Docker** is a system that provides pre-configured, self-contained applications, frameworks, and software stacks, such as WordPress, Golang, or LAMP. Even entire Linux distributions can be run in Docker. When deployed, these software packages are referred to as *containers*. Docker also allows you to create your own containers that include any custom software you'd like. - -**Docker Compose** is a complementary system which helps you link together individual Docker containers so they can work together. This guide walks through the deployment of a Drupal container and another PostgreSQL container that Drupal will use to store its data. Docker Compose will facilitate the networking between them. - -Containers for Drupal and PostgreSQL are available from [Docker Hub](https://hub.docker.com/) in the form of *images*. A Docker image is a static snapshot of a container which is used to create new container instances. Docker Hub is an official repository where individuals and organizations can upload Docker images for public consumption. - -## Why Use Docker to Run Drupal? - -Using the Drupal and PostgreSQL images from Docker Hub offers the following benefits: - -- The configuration of the software has been done for you, which means that you don't need to follow a step-by-step process for each application to get them running on your system. -- Updating your software is as simple as downloading the latest images from Docker Hub. -- Images and containers are self-contained, which means that they are easy to clean up if you decide to remove them. - -## Installation - -### Install Docker - -{{% content "installing-docker-shortguide" %}} - -### Install Docker Compose - -{{% content "install-docker-compose" %}} - -## Set Up Drupal - -1. Create a new directory in your home folder called `my_drupal` and `cd` into it: - - mkdir ~/my_drupal/ - cd ~/my_drupal/ - -2. Create a file named `docker-compose.yml` in this folder and add the following contents. Set your own password for the `POSTGRES_PASSWORD` option. - - {{< file "docker-compose.yml" yaml >}} -version: '3.3' - -services: - drupal: - image: drupal:latest - ports: - - 80:80 - volumes: - - drupal_modules:/var/www/html/modules - - drupal_profiles:/var/www/html/profiles - - drupal_themes:/var/www/html/themes - - drupal_sites:/var/www/html/sites - restart: always - - postgres: - image: postgres:10 - environment: - POSTGRES_PASSWORD: your_postgres_password - volumes: - - db_data:/var/lib/postgresql/data - restart: always - -volumes: - drupal_modules: - drupal_profiles: - drupal_themes: - drupal_sites: - db_data: -{{< /file >}} - -3. From the `my_drupal` directory, start your Docker containers: - - docker-compose up -d - -4. The Docker containers will take a minute or two to start up Drupal and PostgreSQL. Afterwards, you can visit your Linode's IP address in your web browser and you should be directed to the Drupal setup form. - - ![Drupal setup screen in the web browser - 'Choose language' page](drupal_setup_choose_language.png "Drupal setup screen in the web browser - 'Choose language' page") - -5. On the *Set up database* page, select `PostgreSQL` as the *Database type* and enter the following values: - - - Database name: `postgres` - - - Database username: `postgres` - - - Database password: *The password you set in the docker-compose.yml file* - - - Host (under Advanced Options): `postgres` - - ![Drupal setup screen in the web browser - 'Set up database' page](drupal_setup_set_up_database.png "Drupal setup screen in the web browser - 'Set up database' page") - -6. Complete the other screens in the setup guide. When creating your Drupal user, be sure to enter a password that is different from your PostgreSQL password. - -## Usage and Maintenance - -You do not need to manually start your containers if you reboot your Linode, because the option `restart: always` was assigned to your services in your `docker-compose.yml` file. This option tells Docker Compose to automatically start your services when the server boots. - -To stop your Drupal application: - - cd ~/my_drupal/ - docker-compose stop - -This will stop the running Drupal and PostgreSQL containers, but will not remove them. - -To restart your Drupal application: - - cd ~/my_drupal/ - docker-compose start - - -To stop and remove containers, networks and images created by the `docker-compose.yml` file: - - cd ~/my_drupal/ - docker-compose down - -When a Docker container is taken down, it is also deleted; this is how Docker is designed to work. However, your Drupal files and data will be preserved, as the `docker-compose.yml` file was configured to create persistent volumes for that data. - -If you want to remove this data and start over with your Drupal site, you can add the `--volumes` flag to the previous command. **This will permanently delete the Drupal customizations you've made so far.** - - docker-compose down --volumes - -### Update Drupal - -The `docker-compose.yml` specifies the `latest` version of the Drupal image, so it's easy to update your Drupal version: - - docker-compose down - docker-compose pull && docker-compose up -d - -## Next Steps - -More extensive documentation on Docker is available in the [Containers](/cloud/guides/applications/containers) section of the Linode Guides & Tutorials site. diff --git a/docs/guides/quick-answers/linux/linux-vs-windows/index.md b/docs/guides/quick-answers/linux/linux-vs-windows/index.md deleted file mode 100644 index b910bf594d5..00000000000 --- a/docs/guides/quick-answers/linux/linux-vs-windows/index.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -slug: linux-vs-windows -title: "Linux vs. Windows: What’s the Difference?" -title_meta: "What is the Difference Between Linux and Windows?" -description: 'This guide discusses Linux vs. Windows, the difference between Linux and Windows operating systems, and which is best for your needs.' -authors: ["Tom Henderson"] -contributors: ["Tom Henderson"] -published: 2022-04-01 -keywords: ['windows vs linux ','linux or windows','is linux better than windows'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -Users, system administrators, developers, and system architects debate the question of whether Windows or Linux is better for any given purpose. The answers are well-reasoned, full of prejudice, and usually arrive from practical experience. This guide covers the differences between Linux and Windows operating systems and servers. It primarily focuses on the areas of differences between desktops, licensing models, web service technology, developer support, and more. - -## A History of Windows and Linux - -The difference between Linux and Windows begins with their history and evolution. At one time, Windows was the dominant desktop operating system, in an era where the desktop was the context to most computing. The advent of Local area networks (LANs), the Internet, and the World Wide Web influenced the evolution of Windows. - -Rather than being an MS-DOS application, Microsoft Windows 95/98/ME eventually became an operating system when [Windows NT](https://en.wikipedia.org/wiki/Windows_NT) arrived. Windows was molded from a desktop operating system into a server operating system, which led to a high degree of common codebase for desktop and server versions. - -Linus Torvalds, in contrast, developed Linux as an experiment based on [Minix](https://en.wikipedia.org/wiki/Minix). Minix was a variant of the server operating system, Unix. - -Torvalds paired his operating system kernel together with free and open source software (FOSS) provided by the volunteers behind the [GNU software utilities](https://en.wikipedia.org/wiki/GNU_Core_Utilities). Each distribution of Linux was differentiated by their software contents. Linux later added more user-focused applications, graphical user interfaces, and a branch of Linux, named Android. Android now dominates the smartphone world. - -Windows moved away from the DOS command line use of MS-DOS, and Linux slowly added a graphical user interface (GUI), usually as an option presented in various distributions. Windows gained considerable ground for desktop users as the Windows GUI and the number of applications built for it grew and gained acceptance. Windows Desktop versions still statistically dominate world desktop computing. - -## Windows vs. Linux: Desktops - -Today, Microsoft Windows dominates business desktop and laptop computing, with Apple close behind, and Linux lagging in third place. Microsoft has widespread corporate/business use market share based on historical dominance coupled with varied business systems administration software and office applications. These applications and their administration are turning towards cloud infrastructure, largely dependent on Microsoft’s own Azure cloud. - -Linux desktops permit cloud use, and are focused on client-side computing resources. Popular applications that look and feel like Office, do not need hosting or licensing costs, and retain large compatibility with dominant Microsoft offerings. - -Business use of Linux on the desktop can authenticate to Microsoft directory structure resources (Active Directory authentication via Samba). Linux lacks variety in specific areas of application support, but only rarely is there no FOSS-equivalent to a popular genre of Microsoft desktop application. - -Microsoft also ties security to a Trusted Platform Module (TPM) hardware module and the Prism chipset that serves as a core moderator of cryptography and elements of security authentication. Numerous encryption and authorization features must have the TPM hardware available in desktop and notebook resources. - -Linux desktops use alternative encryption methods for files, and a variety of security authentication schemes. Frequently, they use multi-factor authentication devices, like the OAUTH and FIDO2 devices, such as Yubikeys from Yubico. - -### Licensing Differences Between Linux and Windows - -When it comes to licensing and comparing Linux vs Windows, Linux wins for simplicity and zero cost, although support is additional. Licensing in Linux is vastly simpler– it’s free, subject to the terms of the [GNU General Public License (GPL)](https://www.gnu.org/licenses/gpl-3.0.en.html). By contrast, Windows licensing can be complex. Each version also has gradients of feature locks and a purchase gradient tied to its Edition. Edition variants include Starter, Home, Pro, Enterprise, and Educational in both generic and OEM-specific licensing schemes. The cost of each license is buried into the cost of an OEM license through purchase or lease of a desktop, notebook, or device. - -For decades, most desktops arrived with a Windows license, but only recently has Linux been an available option on desktop and notebook hardware. It’s now offered by the top-tier desktop manufacturers, and is supported and issued with Linux and apps onboard. - -Microsoft Windows does not publish the source to the operating system kernel, or most of its utilities, and accompanying apps. Modifying them must be done at the machine code level, or they must be attached via Microsoft’s published application programming interfaces (APIs). Where feasible, API compatibility is offered for communications tasks like web browsing, reading email, transferring files, and Internet tasks. - -## Evolving GUIs: Linux vs. Microsoft - -Most GUIs find their origin in the [Smalltalk programming language](https://en.wikipedia.org/wiki/Smalltalk), along with Windows GUI, macOS, and X Windows roots. In the race between Windows and Linux, Windows continues to improve its GUI software, and developers are familiar with Windows GUI programming. Microsoft makes up for scripting and macro execution by adapting its PowerShell language and APIs to match the rich scripting infrastructure in Unix variants, in this case, Linux. - -Linux functions on 32-64 bit platforms and is less captive to different hardware platforms, providing consistency in GUI and compatibility across a wide variety of desktop hardware, including Intel, AMD, ARM, RISC-V, and other families. Older Linux versions lack many of the applications that made Windows popular. This is a problem now largely solved. Graphics, video editing, CAD, and other applications can be found on Linux using various GUIs. - -Today, it’s unusual to find a desktop or server platform that can’t run either Windows or a Linux distribution, interchangeably, although Microsoft deprecates system compatibility far more frequently than Linux distributions. - -Linux does not have a hardware compatibility mandate for drivers, and until recently, few hardware manufacturers felt the need to develop hardware drivers for a small population of Linux desktop users. Random hardware incompatibility issues dogged Linux on the desktop until the late 2010s. Peripherals vendors initially wrote drivers for their hardware for Windows and perhaps macOS, with Linux as an afterthought, but this trend no longer continues. - -Linux is a kernel and set of apps in a distribution, and there is no central command or business initiative that drives hardware compatibility. Instead, several Linux distribution makers and vendors, use Red Hat or Debian kernel stream versions of Linux to build desktop compatibility, and spearhead driver development for the desktop market. These include Fedora, Alma, and SUSE. Distributions built atop Debian, include Ubuntu, Linux Mint, and several others. - -Although Linux users have been waiting for the “Year of the Linux Desktop”, Microsoft continues to dominate with Windows, despite the licensing costs, foibles, and emblematic failures. - -## Web Server Technologies: Windows vs. Linux - -Linux dominates Windows when it comes to web services, thanks to the success of the [Apache web server](/cloud/guides/how-to-install-apache-web-server-ubuntu-18-04), the [Docker container runtime](/cloud/guides/installing-and-using-docker-on-ubuntu-and-debian), the [Kubernetes container orchestration system](/cloud/guides/beginners-guide-to-kubernetes), and [WordPress publishing platform](/cloud/guides/how-to-install-wordpress-ubuntu-2004). - -Web hosting and other internet services are changing. Microsoft server editions, which can be expensive, host both non-Microsoft web server products as well as its own products. Microsoft attempts to make its offerings including client-side browsers, browser support, web services programming, and hosting models, highly proprietary. - -Linux dominates web services partially because of the [LAMP stack (Linux, Apache, MySQL, and Perl/PHP)](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-20-04). Today, the LAMP stack and variants are a systematic, highly-deployed, go-to development stack. Many of the word's web applications use variants of the LAMP stack to server their content to the Internet. - -Both Apache and [NGINX web servers] dominate the world-wide web, far eclipsing Microsoft’s Internet Information Server/IIS. Both Apache and NGINX FOSS web servers run freely on Windows Server platform, and do so at a highly-scalable and license-cost-free model. - -When compared to Linux's web services, Microsoft’s does not effectively compete. Depending on the licensing situation with Microsoft, web services hosting is often done on FOSS platforms, principally but not exclusively, on Linux for its nominal cost. Although code is free, support is contracted or self-sourced. Microsoft attempts to compete with this trend with its own cloud hosting platform capable of using its [Active Directory services](https://en.wikipedia.org/wiki/Active_Directory). - -When considering Linux vs Windows in the domain of web services and development, Linux wins. Linux statistically has a dominant number of hosted web installations utilizing Apache or NGINX web servers. - -## Developers: Linux vs. Windows - -There is an aphorism that Windows is a developer’s day job, and at night, moonlighting combines Linux and FOSS development. Microsoft now supports Linux development environments with Virtual Machines and desktop guest services. - -Microsoft developer services target platforms for developers including desktops, servers, mainframes, IoT, smartphones, and tablets. Although ports to certain ARM configurations exist, Microsoft Windows primarily ports to numerous IoT platforms and families, as well as OEM service platforms. These are statistically a small amount and often captive to AMD and Intel CPU families and chipsets. - -By contrast, Linux supports kernel ports to IBM Mainframes, RISC-V, updates over long lifecycles to older Intel/AMD platforms. It also ports to SCADA platforms, and industrial configurations. - -Microsoft’s developer network is huge. It supports many aging developer frameworks, subject to current hardware compatibility with the Windows current OS version. The number of APIs in support has meant that Microsoft inadvertently breaks code during updates, patches, and fixes that cannot be tested across the numerous platforms that it supports. - -Microsoft’s internationalization of language support is strong, but because Linux has worldwide development support, porting to target language families through character set support has been part of the Linux experience longer than it has been for Microsoft. - -The most popular Linux distributions have strong documentation to support developers and provide internationalization and multilingual support. Leading Linux distributions, use unique patch, fix, and update software models, which are largely interoperable between distributions.These distributions include Red Hat, SUSE, and Ubuntu. Microsoft provides limited Ubuntu support. Ubuntu can live inside a Windows sandbox, and is progressively interactive with Windows resources. - -Linux vs Windows programming language comparison finds Linux leading across multiple languages and language frameworks. Programming language support in Microsoft’s Visual Studio traditionally supported only Windows hosts as targets until recently. Visual Studio is now ported to Linux, and native Linux language support is gaining wide acceptance. Because of its lightweight OS payload, Linux is also a strong choice fro virtualized hosting and cloud development. Many language and development frameworks are offered on Linux environments, because so much web application development takes place on Linux, rather than Windows. - -In the cloud, there are two camps. The first camp uses Microsoft and Azure where Windows and Linux support is strong. The second camp uses Linux-based hosting. Microsoft Windows hosting is rarer primarily because of the Windows OS payload size, and its cost. - -## Windows and Linux Process Sandboxing - -Keeping processes from interfering with each other, or breaking security models is important. In early days, application sandboxing chiefly took place on [Solaris](https://en.wikipedia.org/wiki/Oracle_Solaris), before the concepts were ported to Linux. Linux now leads Windows in process sandboxing methods. - -Virtual Machines (VMs), which are multiple concurrent instantiations of an OS within a single server, evolved to help partition Windows applications. This was costly, and a branch of server computing emerged where server farms containing virtual machine instances became popular for efficiency, but costly for licensing. - -Windows servers were once tied one-to-a-physical server, but OS virtualization enabled numerous discrete licenses to be hosted within a single server hardware host platform. VMware and others found methods to optimize server licenses and applications instances to squeeze the maximum amount of work from hardware platforms to suit changing needs. Microsoft developed its own virtualization methods. - -Application security and resource controls, or application containers advanced very quickly under first Solaris, then BSD and Linux. Containers, or the sandboxing of application services platforms, is popular for its ability to package and rapidly scale workflows, especially using web-based technologies. The success is mostly due to Docker, a packaging and communications resource allocation methodology. Microsoft ported the concept to Windows server, where it didn't take hold. - -Highly extensible farms of Docker-packaged applications are managed by orchestration, and the Google model of Kubernetes is extremely popular as an orchestration platform for services packaging, deployment, and life-cycle management. When considering Linux vs Windows, Linux wins the battle of rapid, low-cost extensibility. Containers on Linux are not as expensive as discrete instances of virtualized Windows licenses, and their larger, expensive payloads. - -## Similarities Between Windows and Linux - -Both platforms have powerful command line utilities to control desktops, servers, networking, storage, and file systems. These utilities can be executed individually, or through batch or shell-scripting languages. - -Microsoft does little to support external operating system connectivity except through browsers, and only offers the barest Unix or Mac support. Linux is the reverse of this policy. Linux can connect to Windows proprietary extensions of directory services, its Active Directory, through FOSS support from the [Samba project](https://en.wikipedia.org/wiki/Samba_(software)). As both Linux and macOS are branches of the Unix tree, Linux support for macOS and vice-versa is strong. Microsoft and Linux connectivity today are comparatively equal although Microsoft’s halting support of other platforms caused a loss in the battle between Linux vs Windows. - -Microsoft has finally enabled both Linux and macOS to use its proprietary filesystem, NTFS. Both Linux and macOS use a variety of host and target-based filesystems. macOS is free, but its features are largely dedicated to Apple-specific hardware. Linux distributions take pride in working across a wide variety of platforms, without the proprietary chipset support required by Microsoft Windows. - -## Linux vs Windows: Which One is Better? - -Microsoft's closed-source model means that its source code cannot be scrutinized or modified by outside developers. While Linux's open source model means that developers can review and improve a distributions source code. Linux possesses a significant advantage over Windows because of its open source code. Microsoft is growing in its support of open models, but it’s still largely based on proprietary licensing, closed source, and closed model licensing. - -Linux's use cases and connectivity options are publicly published. So are its applications, with few exceptions. Linux distributions remain free of cost, and like Windows code, is patched/updated/fixed on an ongoing basis. Some are patched on a rolling basis, rather than every thirty days, similarly to Microsoft's policy. There are, of course, paid Linux support options too. Many Windows users fear the monthly [*Patch Tuesday*](https://en.wikipedia.org/wiki/Patch_Tuesday) for its unknown consequences when updates are applied. - -Many eyes on code increases code quality, the rapidness of bug fixes, security patches, and the ability to mate applications together safely. This is an aspect of Linux operating systems. However, like most things related to computing and web services, the best option between Windows and Linux largely depends on your particular use case and the type of IT resources that you have available. - diff --git a/docs/guides/quick-answers/linux/log-in-to-coreos-container-linux/index.md b/docs/guides/quick-answers/linux/log-in-to-coreos-container-linux/index.md deleted file mode 100644 index 9b4bdd7eb29..00000000000 --- a/docs/guides/quick-answers/linux/log-in-to-coreos-container-linux/index.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -slug: log-in-to-coreos-container-linux -title: Log in to CoreOS Container Linux -description: 'This quick answer guide will show you how to log in to CoreOS Container Linux.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2017-05-08 -keywords: ["coreos", "container linux", "login", "core"] -tags: ["container","linux"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- - -The `root` user is not active by default in Container Linux, so root login is not available. Instead, log in as the `core` user. - -## Log in to CoreOS Over SSH - -![Log in as core user SSH](container-linux-login-ssh.png) - -1. From your local terminal, enter: - - ssh core@ - -2. At the `password:` prompt, enter the `core` user's password you assigned when first having deployed Container Linux. - -## Log in to CoreOS Through the Console (Lish or Glish) - -![Log in as core user Lish](container-linux-login-lish.png) - -1. At the `login:` prompt, enter `core`. - -2. At the `password:` prompt, enter the `core` user's password you assigned when first having deployed Container Linux. - -{{< note >}} -If you are not already familiar with the serial and graphical Linode shells, see the [Using the Lish Console](https://techdocs.akamai.com/cloud-computing/docs/access-your-system-console-using-lish) and [Access Your Linux Desktop Using Glish](https://techdocs.akamai.com/cloud-computing/docs/access-your-desktop-environment-using-glish) guides. -{{< /note >}} diff --git a/docs/guides/quick-answers/linux/open-source-vs-closed-source/index.md b/docs/guides/quick-answers/linux/open-source-vs-closed-source/index.md deleted file mode 100644 index 514d7de084c..00000000000 --- a/docs/guides/quick-answers/linux/open-source-vs-closed-source/index.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -slug: open-source-vs-closed-source -title: "Open Source vs. Closed Source: What’s the Difference?" -title_meta: "What is the Difference Between Open Source and Closed Source?" -description: 'Comparing open source vs. closed sourced software? Discover the definition for each and the differences that define the two terms.' -authors: ["Steven J. Vaughan-Nichols"] -contributors: ["Steven J. Vaughan-Nichols"] -published: 2022-01-21 -keywords: ['open source versus closed source','open source vs. closed source','difference between open source and closed source '] -tags: ['linux'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -## What is Open Source? - -In the beginning, back when Remington Rand’s Univac 1, the DEC PDP-1, and IBM System 360 ruled the computing world, all software was *open-source* software. This means that anyone could take the source code and modify and improve it to meet their own needs. Early software was created by government agencies and schools. In the interest of getting programs up and running they’d share their source code either with programmer to programmer swaps or via user groups such as IBM Share and DEC DECUS. - -As time passed, several events came together, which led to the rise of proprietary software. First, as computers became more standardized, original equipment manufacturers (OEMs) began bundling software with their hardware. Typically the price of the software was included with the computers. Then, 1969’s United States vs. IBM antitrust suit ruled that it was anticompetitive for IBM to give away its software. This was followed by the 1974 US Commission on New Technological Uses of Copyrighted Works (CONTU) which decided that computer programs could be copyrighted. Combine this with the growing cost of developing software, and conditions were ripe for a new kind of business: Selling closed-source, proprietary software. - -During the 1950s through the early 70s, source code was always included with an operating system or program, and users were only given binaries. Since then, if you ask most people what a program is they describe something that requires a mouse click, a touch, or at most, entering a single word command to run. - -That’s fine for users. For programmers, it was another story. Instead of sharing the load of developing software, they had to reinvent the wheel over and over again as each company pursued its own way of solving common development problems. - -One developer who was especially annoyed at closed-source software was MIT’s Richard M. Stallman. He created a new legal approach, the GNU Public Library (GPL) for what he called “Free Software.” This framework would prove the launch-pad for open-source software. - -Over the last 30-years, open-source software made a comeback. First, the rise of the commercial Internet and web made it easier for developers to work together. Then, open-source programs such as the Linux operating system and the [Apache HTTP Server](/cloud/guides/web-servers/apache-tips-and-tricks) showed that this kind of software could be far cheaper, better, and more secure than their closed-source rivals. - -Since then, many popular end-user programs such as Firefox, LibreOffice, and Thunderbird often replace, or rival, closed-source programs such as Internet Explorer, Microsoft Office, and Outlook. - -While open-source software is successful for desktop users, where it really shines is in server and service software. Today, even on the Microsoft Azure cloud, Linux, not Windows Server, is the most popular operating system. Almost all cloud computing services operating systems, Infrastructure-as-a-Service (IaaS), Software-as-a-Service (DaaS), and databases, are based on open-source software. Without open-source software, there would be no cloud computing. - -### Open Source Pros and Cons - -Why is open-source so successful? It starts with developers wanting to improve the programs they use. As Eric S. Raymond puts it in his seminal open-source work, The Cathedral and the Bazaar, "Every good work of software starts by scratching a developer's personal itch." - -It’s more than personal though. By improving any program, everyone who uses it gets a better version. Open-source developers and companies instead of competing with each other, cooperate with each other. The idea is that a rising sea makes all ships rise. They still compete with each other, but if everyone’s software is fundamentally better, everyone benefits. Except, of course, for those building closed-source software. - -Besides improving quality, this cooperation also drops the cost of developing software. If you and twenty other people from five different companies are all working on the same problem, every business cuts their programming costs. - -Generally speaking, open-source software is more secure than its closed-source rivals. That’s because, as Raymond puts it in what he called Linus’s Law, "given enough eyeballs, all bugs are shallow." - -Open-source software also has the advantage that it never runs out of support. A company can go out of business, but the code is still available, and if there’s enough interest in the program someone else picks it up and keeps it going. If you have skilled programmers on staff, you can do it yourself. - -Last, open-source software is cheap. If you don’t need support, it’s often free. - -There are very few problems with the open-source approach. In the case of some less well-known programs, you may have trouble finding end-user support. - -In some cases, you may find it hard to find and fix problems with open-source programs. That’s because many developers do a poor job of reporting on exactly what open-source code they use in their programs. To combat this problem, open-source groups support Software Bills of Material (SBOM). An SBOM spells out exactly what software libraries, routines, and other code has been used in any program. Armed with this, you can examine what component versions are used in your program. Of course, with closed-source programs, you never know what went wrong. - -## What is Closed Source? - -Closed-source software is the exact opposite of open-source software. Instead of having access to the source code, all you get is the black box of the binary code. - -Some of the most well-known programs written in closed source include Microsoft Office, games such as Fortnite, Call of Duty, and Grand Theft Auto, and essentially all Apple software. - -Despite the benefits of open-source approaches, the vast majority of personal and end-user software is written with proprietary code. It’s an entirely different story when it comes to server-level and service programs. There, open-source software rules. - -### Closed Source Pros and Cons - -For its makers, closed-source software is more easily profitable. They sell, or increasingly rent you a subscription, you use the program, and they get paid. - -When you get a closed-source program, you get a single vendor to turn to for all your support needs. - -Generally speaking, closed-source programs have better interfaces. Because, unlike open-source software, they expect you to pay in advance for their program. For this reason, the software should look as good as possible and have well-thought out user experiences. - -On the other hand, closed-source is more insecure than its open-source brethren. For example, every month Microsoft must release new security fixes, Patch Tuesday, for all its programs' most recently discovered flaws. - -Closed-source software is also more expensive than open-source software. There are almost no free closed-source programs. - -Once you’re using a closed-source program, you have few, if any, options to modify it for your specific needs. What you see is what you get. - -## The Difference between Open Source and Closed-Source Software - -These are two fundamentally different approaches to creating software. Here are the important differences: - -1. You can modify open-source programs. You can’t change closed-source ones. -1. Open-source software tends to be free or inexpensive. Closed-source software is more expensive. If you must pay for open-source software support, it may be less expensive. -1. There are no user or CPU licensing restrictions on open-source software. Closed-source programs always have user and/or CPU licensing fees and restrictions. -1. Generally speaking, open-source software is more secure than closed-source programs. But, security is not a product, it’s a process. Both kinds of programs are vulnerable to attackers. -1. Open-source programs are constantly evolving and improving. Closed-source ones tend to change at a slower pace. If you’d rather stick with the tried and true, many major open-source programs have long-term support (LTS) versions, so you don’t need to be constantly updating them. -1. Closed-source programs may go out of date so you have no choice but to upgrade to another version. For example, Windows 7 users had to migrate to Windows 10. Open-source programs may go out of date, but you can continue to run older versions. -1. Unless you buy a support contract for an open-source program, you’re on your own. There is, however, free community support for most programs. With closed-source programs, some support is usually bundled into the price. -1. With open-source programs it can be hard to determine who’s responsible for patching your software. With closed-source software, you almost always have a single vendor to talk to if you run into a problem. - -## Conclusion: Open-source Software for the Win - -There are some occasions when closed-source software is better. For example, if you use Adobe Photoshop extensively in your business, for better or worse, you’re going to be using this closed-source program. While there is a good open-source graphical editor, GIMP, it doesn't have Photoshop’s vast software ecosystem. - -When you’re using the cloud, open-source software runs the world. There are some proprietary programs that run off the cloud, such as Microsoft 365. Except for these end-user programs, everything else worth considering is based on open source. - -Linode knows this well. Linode’s Open Cloud is 100% based on open-source programs and protocols. For example, the Linode application programming interface (API) is completely open and free. You can build your apps and infrastructure on top of it with confidence and without sacrificing your freedom. - -While all cloud providers rely on open-source software, Linodes support all open-source distros. The company doesn't offer closed-source forks. - -Linode doesn't just make use of open source. They’re open-source developers as well. Linode’s Manager, CLI, and product and API documentation are all on [Github](https://github.com/linode/) and open source. diff --git a/docs/guides/security/authentication/oauth2-authentication-an-introduction/index.md b/docs/guides/security/authentication/oauth2-authentication-an-introduction/index.md deleted file mode 100644 index c85ab2d6657..00000000000 --- a/docs/guides/security/authentication/oauth2-authentication-an-introduction/index.md +++ /dev/null @@ -1,240 +0,0 @@ ---- -slug: oauth2-authentication-an-introduction -title: "What Is OAuth2 Authentication?" -title_meta: "An Introduction to OAuth2 Authentication" -description: 'This guide provides an introduction to Oauth2 authentication, flows, scopes, and libraries.' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2021-10-22 -keywords: ['What is Oauth2','How does Oauth2 work','Oauth2 flow','Oauth vs oauth2'] -tags: ['python', 'security'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[OAuth2 website](https://oauth.net/2/)' -- '[OAuth2 libraries](https://oauth.net/code/)' ---- - -[*OAuth 2.0*](https://oauth.net/2/) is an authorization protocol that helps application users securely share access to their accounts. For example, with the help of OAuth 2.0, a social media app user can securely share their email contacts with the app. Providing account details directly to the application is highly problematic from the perspective of web security and privacy. OAuth2 allows users to grant limited access to their accounts with third-party applications without sharing any passwords. This guide discusses how OAuth2 works and compares it to other authorization frameworks. - -## What is OAuth2? - -OAuth release 2.0 has become the tech industry's open standard for authorization and access delegation. It allows for some degree of flexibility and leaves many decisions up to the individual implementations. Security flaws have been discovered in some of these libraries, but OAuth2 best practices can reduce the risk of these problems. - -The OAuth process allows users to authorize web applications to access their accounts without sharing login or password details. Authorization details are handled by the site hosting the account, not the site requesting the access. This is why OAuth is known as an authorization protocol, not an authentication protocol. This process is also known as *secure delegated access*. OAuth is application-centric. This means permissions are granted to a specific client application rather than being attached to a particular user. - -The service uses OAuth to dispense *access tokens* containing account permissions to third-party applications. This token can then be used to directly access the actual account information. OAuth is specifically designed to work with the *Hypertext Transfer Protocol Secure* (HTTPS) protocol. It uses the *Secure Sockets Layer* (SSL) to ensure data and tokens remain safe during transmission. - -## Benefits of OAuth2 Authentication - -OAuth2 features some advantages over other more complex protocols. The current release has evolved and matured in comparison to the original OAuth protocol. Some advantages include the following: - -- OAuth's token-based method is more secure than sharing password details directly with third-party applications. -- It gives the user more control over what information they are sharing. -- OAuth2 allows users to revoke their access tokens if they want to disable access. -- It is supported and recommended by many large tech organizations such as Facebook, Microsoft, and Google. Some of these sites require the use of OAuth 2.0 to access account information. -- OAuth2 can be used to access secure RSS or ATOM feeds, which had been difficult to access before. -- OAuth2 is flexible and can be used by web, desktop, and mobile applications. -- It is compatible with *Single Sign-On* (SSO) systems and authentication protocols. -- It is relatively easy for developers to implement. Many third-party OAuth libraries are available. - -### OAuth vs OAuth2 - -OAuth1 was originally based on *Flickr's authorization protocol* and *Google's AuthSub* utility. OAuth2 completely overhauled the first release of OAuth and should be thought of as a completely new protocol. Here are some of the major differences between the two releases: - -- OAuth1 and OAuth2 are not compatible. -- OAuth2 is designed to simplify some sections of the original protocol that developers found confusing. -- OAuth1 uses cryptographic techniques and digital signatures to verify message integrity, whereas OAuth2 bases its authorization mechanism around access tokens. Cryptographic techniques are more secure, but also more difficult to implement. However, tokens can be copied or even stolen under certain circumstances. -- OAuth2 uses HTTPS and TLS to handle security, whereas OAuth1 was more protocol agnostic. This means release 2.0 is more dependent on its transport protocols. -- The tasks of account verification and resource handling are decoupled in OAuth2. -- OAuth2 is more flexible than OAuth1, and can now handle applications other than web clients. - -Overall, there is a tradeoff between the two releases. OAuth2 is easier to use but is slightly less secure. - -## OAuth2 Definitions - -The OAuth2 protocol introduces a substantial amount of new terminologies. Some of the key terms are described in this section. - -### OAuth2 Scopes - -In OAuth2 terminology, a *scope* specifies the level of access the client is requesting. It refers to the information or functionality that the application can access, and whether read-only or write access is allowed. The particular data contained within a scope is referred to as a *claim*. - -### OAuth2 Roles - -There are four roles defined within the OAuth 2 authorization flow. These roles describe an actor's position within the protocol and the actions they might take. - -**Resource Owner:** This role refers to a person or service that grants access to certain aspects of their account. In a typical workflow, this is a user who logs in to one of their accounts via a third-party application. As part of the authorization process, the resource owner can specify a scope to apply to the request. - -**Client:** This is the application that is attempting to access the resource owner's account. However, it must obtain permission from the owner before it can make its request. In most cases, clients must pre-register with the target service and obtain a designated client identifier before making any authorization requests. The client must present these credentials when attempting to access any account information. - -**Authorization Server:** This server validates the user credentials and provides an access token that can be used to request the resources at some later point. - -**Resource Server:** This server hosts the account resources that the client wants to access. Clients must use the resource server's API and submit an access token when making a request. In some cases, this might be the same device as the authorization server, but for popular services, they are usually different. - -## How does OAuth Work? A Description of the OAuth2 Authentication Flow - -Although the exact OAuth2 flow differs somewhat depending on the application, there are three main sections of the authorization procedure. The approach described in this section is recommended for web server applications where processing takes place internally and code is not externally visible. A brief explanation of single-page or JavaScript applications where the source code is exposed follows the main section. - -### Step 1: Client and Resource Owner - -In this step, the client asks the resource owner for access and presents them with an authorization request. For example, the web application could provide the user with, or redirect a user to, a URL where they can enter authorization details. The resource owner can then decide whether or not to grant access and specify the scope for the access. The user typically authorizes the request by entering either their account details or a one-time code. - -If the authorization request is successful, the target service transmits an authorization code back to the client, typically as part of the redirect URL. The application stores this code for use in the next step of the negotiation. - -### Step 2: Client and Authorization Server - -When the client receives the code, it asks the service's authorization server for an *access token*. As part of the request, it supplies the authorization code and proof of its own identity. It typically transmits its request in the form of an HTTPS message. As soon as the server validates the information and verifies the client is trustworthy, it sends the client an access token. The client never learns anything about the resource owner's account information. - -### Step 3: Client and Resource Server - -In the final phase of the negotiation, the client presents the access token to the resource server, and requests access to the protected resource. If the access token is still valid, the server provides the client access to the account within the scope of the access grant. - -{{< note >}} -Single-page applications, including those generated entirely through JavaScript, cannot securely maintain a secret client identity. In this case, the *Proof Key for Code Exchange* (PKCE) extension is used to dynamically generate a secret key for each request. There are some additional concerns for mobile applications. All applications can use PKCE to eliminate the possibility of the code being intercepted and to enhance security. Consult the [*OAuth2 specification*](https://oauth.net/2/) for more details. - -Devices without a keyboard, such as smart televisions, typically implement OAuth2 using a *device code*. The device code is used alongside a user code that is submitted elsewhere. -{{< /note >}} - -## Popular OAuth2 Libraries - -Links to popular OAuth2 libraries are available in a variety of languages, including JavaScript, Python, and PHP, courtesy of the [*OAuth2 libraries directory*](https://oauth.net/code/). These libraries are divided into client libraries and server libraries. Client libraries are used by applications to access protected account services. There are also links to open source and commercial OAuth providers. It is best to use a library recommended by the OAuth site because other implementations might have security concerns or coding flaws. - -Due to a large number of libraries, this guide focuses on the high-level task of selecting, installing, and using an OAuth2 client library. In this section, a sample Python library is used as an example. Pseudocode is shown for each step, with a more detailed summary of the code at the end of this section. Some basic knowledge of Python is necessary to understand the code samples. - -{{< note >}} -You must register the application with the service before accessing any account information. You must also register and submit a URL to redirect client traffic. After approving your registration, the application provides you with a client ID and secret. All applications must use HTTPS when transmitting and receiving OAuth2 messages. -{{< /note >}} - -### How to Use a Python OAuth2 Library - -1. Review the supported languages on the [*OAuth2 libraries page*](https://oauth.net/code) and select the category corresponding to the language you are using. Alternatively, select a link to a commercial or open-source provider. - -1. Select a library from the available choices. Evaluate the alternatives by comparing how many downloads and stars each package has received on GitHub. Review the documentation, as some packages are easier to install and use than others. This tutorial uses the [*Rauth*](https://github.com/litl/rauth) package from GitHub for an example. The exact installation instructions depend on the package. - -1. Download and install the library. In many cases, the library can be installed using `apt` or `pip`. In other cases, the library must be added to the source list first. - - {{< note respectIndent=false >}} -The following command assumes Python and `pip` are already installed on the Linode. - {{< /note >}} - - sudo pip3 install rauth - - {{< output >}} -Installing collected packages: rauth -Successfully installed rauth-0.7.3 - {{< /output >}} - -1. Inside the python file, import `OAuth2Service` from the `rauth` package. - - {{< file "oauth2.py" python >}} -from rauth import OAuth2Service - {{< /file >}} - -1. Instantiate an `OAuth2Service` container object for use throughout the authorization process. Use the `client_id` and `client_secret` that were assigned to the application when it was registered. The other values are unique to the service being accessed. Verify all URLs using the service documentation. - - {{< file "oauth2.py" >}} -service = OAuth2Service(params) - {{< /file >}} - -1. Use this object to access the redirect URL for the service. When the client finishes authorizing access to the service, this link contains an authorization code. Consult the [service documentation](https://rauth.readthedocs.io/en/latest/api/#oauth-2-0-services) for details about how the code is embedded in the `url` variable. - - {{< file "oauth2.py" python >}} -url = service.get_authorize_url(**params) - {{< /file >}} - -1. Extract the authorization code from the `url` and use it to request an access token for the user account. Submit the authorization code as part of the `data` object. - - {{< file "oauth2.py" python >}} -token = service.get_auth_session(data=data) - {{< /file >}} - -1. The session can now be used to access account information. - - {{< file "oauth2.py" python >}} -r = token.get(params) -"do something with r" - {{< /file >}} - -1. An actual implementation of this scenario, using methods and parameters from the `rauth` library, would be similar to the following prototype. Substitute the appropriate URLs for the service being accessed in place of the `example.com` URLs. - - {{< file "oauth2.py" python >}} -from rauth import OAuth2Service - -# Initialize the container - -service = OAuth2Service( - name='example', - client_id='123', - client_secret='456', - access_token_url='https://example.com/token', - authorize_url='https://example.com/authorize', - base_url='https://example.com/api/') - -# Construct the params dict and use it to retrieve the url embedded in the authorization code - -params = {'redirect_uri': 'http://example.com/', - 'response_type': 'code'} -url = service.get_authorize_url(**params) - -# Write a routine to extract the code from the url based on the service documentation - -code = parse_url_for_code(url) - -# Construct the data dict and use it to obtain an access token - -data = {'code': 'code_from_url_above', - 'grant_type': 'authorization_code', - 'redirect_uri': 'http://example.com/'} - -token = service.get_auth_session(data=data) - -# Use the token as required - -r = token.get('url', params={'format': 'json'}) - {{< /file >}} - -## OAuth2 Comparisons - -OAuth is not the only protocol that can be used for authentication and authorization. Some competing and complementary protocols are [*OpenID*](https://openid.net/) and [*OpenID Connect*](https://openid.net/connect/) (OIDC), *eXtensible Access Control Markup Language* (XACML), and the *Security Assertion Markup Language* (SAML). - -OAuth2 can be used for authentication without an assisting protocol. This process is known as *pseudo-authentication*, but it is not recommended. - -### OAuth2 vs OpenID/OIDC - -OpenID is complementary to OAuth2. It is intended for authentication rather than authorization. - -Some of the similarities and differences between OAuth2 and OpenID are as follows: - -- OAuth2 is used for authorization while OpenID is mainly for authentication. Authorization means the user is allowing the application to access an account they own. For authentication, the user must prove their identity. -- An OpenID server used for authenticating a user is typically referred to as an *identity provider*. -- In OpenID, the response from the authorization server affirms the resource owner's identity. Consequently, OpenID does not provide an access token. Instead, it dispenses an *ID token*. The information in this token has a special format and is encapsulated in a tamper-proof *JSON Web Token* (JWT). -- OAuth2 is not specifically a means of authentication, even though the authorization server typically authenticates the user while validating their credentials. - -OAuth2 is more closely related to OIDC, an authentication layer built on top of OAuth2. The two protocols can be used together. A comparison between OIDC and OAuth2 is as follows: - -- OIDC allows clients to validate the identity of a user based on the results of the authorization procedure. -- OIDC can be used to obtain details about the user, including their account profile or log-in details. OAuth does not have any such capabilities. It can access account details without knowing anything about the user. -- Both protocols are fairly flexible and can support web and mobile clients. -- OIDC has additional capabilities. It can discover OpenID Providers and manage session details. It can also be configured to encrypt the identity details. - -### OAuth2 vs XACML - -XACML is an alternative for authorization that is based on access control. However, it can be combined with OAuth2 for a more complete solution. Some of the differences between the two protocols are as follows: - -- XACML uses policies to control the level of access. -- XACML uses a request and response format for its requests. -- XACML is more granular than OAuth is. Its policies can take into account attributes including the context, the action being taken, and the nature of the resource. OAuth has access to the resource based on the scope of the grant, with no further restrictions and no additional access rights. -- OAuth2 requires HTTP, but XACML can work over a wider range of applications, including databases. - -XACML policies can potentially incorporate OAuth2 authorization grants. OAuth2 can obtain the authorization and gain delegated access rights. XACML can then further refine this access based on its control policies. For example, it can determine whether a user has read-only or write access within different ranges of the OAuth2 scope. - -### OAuth2 vs SAML - -SAML is also used for authentication and identity management. It provides tokens containing the user's identification record. SAML is more commonly used in enterprise settings, like Enterprise Single Sign-On (SSO). In an enterprise environment, it gives the user access to a wide range of company resources with only one login. OAuth2 and SAML can both be used for single sign-on, but SAML is user-centric, while OAuth2 provides access to a specific client application. - -OAuth2 and SAML can be used together. However, OAuth2 and OIDC are considered a better combination that is usually easier to implement. - -## OAuth2 Summarized - -OAuth2 is the current standard for authorization and access delegation. It defines several user roles and outlines a procedure for authorization. In a typical OAuth2 flow, the client acquires an access token after a successful authorization exchange. It then uses this token to access protected resources. OAuth2 is easier to implement than the first release of OAuth but is perhaps not quite as secure. There are a few alternatives and complements to OAuth2, including OpenID/OIDC and XACML. - -A large number of OAuth2 libraries are available in different programming languages. These can be found on the [*OAuth2 site*](https://oauth.net/code/). The typical library includes high-level instructions for each step of the authorization process. diff --git a/docs/guides/security/monitoring/visualize-server-security-on-centos-7-with-an-elastic-stack-and-wazuh/index.md b/docs/guides/security/monitoring/visualize-server-security-on-centos-7-with-an-elastic-stack-and-wazuh/index.md deleted file mode 100644 index c796367e3ed..00000000000 --- a/docs/guides/security/monitoring/visualize-server-security-on-centos-7-with-an-elastic-stack-and-wazuh/index.md +++ /dev/null @@ -1,592 +0,0 @@ ---- -slug: visualize-server-security-on-centos-7-with-an-elastic-stack-and-wazuh -title: "Visualize Server Security on CentOS 7 with an Elastic Stack and Wazuh" -title_meta: "How to Visualize Server Security on CentOS 7" -description: "Learn how to use the Elastic Stack to collect, log, and visualize security data and threat alerts through Wazuh, part of OSSEC Intrusion Detection." -authors: ["Andrew Lescher"] -contributors: ["Andrew Lescher"] -published: 2017-10-17 -modified: 2019-01-31 -keywords: ["ossec", "elk stack", "elk,ossec-hids"] -tags: ["monitoring","security","lemp","centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Wazuh Official Documentation](https://documentation.wazuh.com/current/index.html)' - - '[OSSEC Official Documentation](http://ossec-docs.readthedocs.io/en/latest/index.html)' -dedicated_cpu_link: true -aliases: [] ---- - -![Visualize Server Security on CentOS 7 with an Elastic Stack and Wazuh](elastic-stack-security-title.jpg "Visualize Server Security on CentOS 7 with an Elastic Stack and Wazuh") - -## What are Elasticsearch, Elastic Stack, and Wazuh? - -An Elastic Stack, formerly known as an ELK Stack, is a combination of Elasticsearch, Logstash, and Kibana. In this tutorial, you will learn how to install and link together ElasticSearch, Logstash, Kibana, with Wazuh OSSEC to help monitor and visualize security threats to your machine. The resulting structure can be broken down into three core components that work with Wazuh's endpoint security: - -- **Elasticsearch** - - - The heart of the Elastic Stack, Elasticsearch provides powerful search and analytical capabilities. It stores and retrieves data collected by Logstash. - -- **Logstash** - - - Ingests data from multiple sources and passes it along to Elasticsearch which acts as a central database. - -- **Kibana** - - - A self-hosted, web-based tool which provides a multitude of methods to visualize and represent data stored in Elasticsearch. - -## What is Wazuh OSSEC - -Wazuh is an open source branch of the original [OSSEC HIDS](https://ossec.github.io/) developed for integration into the Elastic Stack. Wazuh provides the OSSEC software with the OSSEC ruleset, as well as a RESTful API Kibana plugin optimized for displaying and analyzing host IDS alerts. - -## Before You Begin - -1. Many of the steps in this guide require root privileges. Complete the sections of our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) to create a standard user account, harden SSH access and remove unnecessary network services. Use `sudo` wherever necessary. - -2. Your Linode should have at least [8GB of RAM](https://www.linode.com/pricing). While an Elastic Stack will run on less RAM, the Wazuh Manager will crash if RAM is depleted at any time during use. - -3. Add a domain zone, NS record, and A/AAA record for the domain you will use to access your Kibana installation. See the [DNS Manager](https://techdocs.akamai.com/cloud-computing/docs/dns-manager) guide for details. If you will access your Kibana instance via your Linode's IP address, you can skip this step. - -4. [Create an SSL Certificate](/cloud/guides/install-lets-encrypt-to-create-ssl-certificates), if you will be using SSL encryption for your domain. - -5. Install NGINX or Apache. Visit our guides on how to install a LEMP or LAMP stack for CentOS for help: - - - [Install a LEMP Stack on CentOS 7 with FastCGI](/cloud/guides/lemp-stack-on-centos-7-with-fastcgi) - - [LAMP stack on CentOS 7](/cloud/guides/how-to-install-a-lamp-stack-on-centos-7) - -6. Configure your webserver for virtual domain hosting: - - **NGINX** - - - [How to Configure NGINX](/cloud/guides/how-to-configure-nginx) - - **Apache** - - - [Apache Configuration Basics](/cloud/guides/apache-configuration-basics) - -## Update System and Install Prerequisites - -1. Update system packages: - - yum update -y && yum upgrade -y - -1. Install Java 8 JDK: - - yum install java-1.8.0-openjdk.x86_64 - -1. Verify the Java installation by checking the version: - - java -version - - Your output should be similar to: - - ```output - openjdk version "1.8.0_191" - OpenJDK Runtime Environment (build 1.8.0_191-b12) - OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode) - ``` - -1. If your Linode doesn't have curl installed, install curl: - - yum install curl - -## Install Wazuh - -1. Create the `wazuh.repo` repository file and paste the text below: - - {{< file "/etc/yum.repos.d/wazuh.repo" >}} -[wazuh_repo] -gpgcheck=1 -gpgkey=https://packages.wazuh.com/key/GPG-KEY-WAZUH -enabled=1 -name=CentOS-$releasever - Wazuh -baseurl=https://packages.wazuh.com/3.x/yum/ -protect=1 -{{< /file >}} - -1. Install Wazuh Manager: - - yum install wazuh-manager - -1. Install Wazuh API: - - 1. Install the Node.js repository: - - curl --silent --location https://rpm.nodesource.com/setup_8.x | bash - - - 1. Install Node.js: - - yum install -y nodejs - - 1. Install Wazuh API: - - yum install wazuh-api - - {{< note respectIndent=false >}} - Python >= 2.7 is required in order to run the Wazuh API. To find out which version of Python is running on your Linode, issue the following command: - - python --version -{{< /note >}} - -## Install Elasticsearch, Logstash, and Kibana - -Install the Elastic Stack via RPM files to get the latest versions of all the software. Be sure to check the [Elastic website](https://www.elastic.co/downloads) for more recent software versions. Adjust the commands below to match. - -### Install Elasticsearch - -1. Download the Elasticsearch RPM into the `/opt` directory: - - cd /opt - curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.2.rpm - -1. Install Elasticsearch: - - rpm -i elasticsearch-6.5.2.rpm - -1. Enable the Elasticsearch service to start on system boot: - - systemctl enable elasticsearch - systemctl start elasticsearch - -1. Verify that Elasticsearch has installed and is listening on port 9200: - - curl "http://localhost:9200/?pretty" - - You should receive a similar response: - - ```output - { - "name" : "-7B24Uk", - "cluster_name" : "elasticsearch", - "cluster_uuid" : "UdLfdUOoRH2elGYckoiewQ", - "version" : { -   "number" : "6.5.2", -    "build_flavor" : "default", -   "build_type" : "rpm", -   "build_hash" : "9434bed", -   "build_date" : "2018-11-29T23:58:20.891072Z", -   "build_snapshot" : false, -   "lucene_version" : "7.5.0", -   "minimum_wire_compatibility_version" : "5.6.0", -   "minimum_index_compatibility_version" : "5.0.0" -   }, - "tagline" : "You Know, for Search" - } - ``` - -1. Load the Wazuh Elasticsearch template. Replace `exampleIP` with your Linode's public IP address: - - curl https://raw.githubusercontent.com/wazuh/wazuh/3.7/extensions/elasticsearch/wazuh-elastic6-template-alerts.json | curl -X PUT "http://exampleIP:9200/_template/wazuh" -H 'Content-Type: application/json' -d @- - - -### Install Logstash - -1. Download the Logstash RPM into the `/opt` directory: - - cd /opt - curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.5.2.rpm - -1. Install Logstash: - - rpm -i logstash-6.5.2.rpm - -1. Enable Logstash on system boot: - - systemctl daemon-reload - systemctl enable logstash - systemctl start logstash - -1. Download the Wazuh config file for a **single-host architecture** for Logstash: - - curl -so /etc/logstash/conf.d/01-wazuh.conf https://raw.githubusercontent.com/wazuh/wazuh/2.0/extensions/logstash/01-wazuh.conf - -1. Add the Logstash user to the `ossec` group to allow access to restricted files: - - usermod -aG ossec logstash - -**For CentOS 6 and RHEL 6 Only:** - -1. Edit `/etc/logstash/startup.options` to change the `LS_GROUP=logstash` to `LS_GROUP=ossec`: - - {{< file "/etc/logstash/startup.options" >}} -. . . -# user and group id to be invoked as -LS_USER=logstash -LS_GROUP=logstash -. . . -{{< /file >}} - -1. Update the service with the new parameters: - - /usr/share/logstash/bin/system-install - -1. Restart Logstash: - - systemctl restart logstash - -### Install Kibana - -1. Download the Kibana RPM into the `/opt` directory: - - cd /opt - curl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-6.5.2-x86_64.rpm - -1. Install Kibana: - - rpm -i kibana-6.5.2-x86_64.rpm - -1. Enable Kibana on system boot: - - systemctl enable kibana - systemctl start kibana - -1. Install the Wazuh app for Kibana: - - sudo -u kibana NODE_OPTIONS="--max-old-space-size=3072" /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.7.1_6.5.2.zip - - The Kibana app installation process takes several minutes to complete and it may appear as though the process has stalled. - -1. By default Kibana only listens on the loopback interface. To configure it to listen on all interfaces, update the `/etc/kibana/kibana.yml` file and uncomment `server.host` and the following value: - - {{< file "/etc/kibana/kibana.yml">}} -# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values. -# The default is 'localhost', which usually means remote machines will not be able to connect. -# To allow connections from remote users, set this parameter to a non-loopback address. -server.host: "0.0.0.0" - {{}} - - Reference the table below for information on other configurations available in the `/etc/kibana/kibana.yml` file: - - | Value | Parameter | - | :-------------: | :----------------------------------------------------------------------------------------: | - | server.port | If the default port `5601` is in use, change this value. | - | server.name | This value is used for display purposes only. Set to anything you wish, or leave it unchanged. | - | logging.dest | Specify a location to log program information. `/var/log/kibana.log` is recommended. | - - You may modify other values in this file as you see fit, but this configuration should work for most. - -1. Restart Kibana: - - systemctl restart kibana - -## Configure the Elastic Stack - -The Elastic Stack will require some tuning before it can be accessed via the Wazuh API. - -1. Enable memory locking in Elasticsearch to mitigate poor performance. Uncomment the `bootstrap.memory_lock: true` line in the `/etc/elasticsearch/elasticsearch.yml` file: - - {{< file "/etc/elasticsearch/elasticsearch.yml">}} -# ----------------------------------- Memory ----------------------------------- -# -# Lock the memory on startup: -# -bootstrap.memory_lock: true -# - {{}} - -1. Edit locked memory allocation. Follow the instructions under the appropriate init system used on your Linode: - - **SystemD** - - Edit the systemd init file and add the following line: - - {{< file "/etc/systemd/system/multi-user.target.wants/elasticsearch.service" >}} -. . . -LimitMEMLOCK=infinity -. . . -{{< /file >}} - - **System V** - - Edit the `/etc/sysconfig/elasticsearch` file. Add or change the following line: - - {{< file "/etc/sysconfig/elasticsearch" >}} -. . . -MAX_LOCKED_MEMORY=unlimited -. . . -{{< /file >}} - -1. Configure the Elasticsearch heap size based on your Linode's resources. This figure will determine how much memory Elasticsearch is allowed to consume. Keep the following rules in mind: - - - No more than 50% of available RAM - - No more than 32GB of RAM - - The `-Xmsg` and `-Xmxg` values must be the same in order to avoid performance issues. - - Open the `jvm.options` file and navigate to the block shown here: - - {{< file "/etc/elasticsearch/jvm.options" >}} -. . . -# Xms represents the initial size of total heap space -# Xmx represents the maximum size of total heap space - --Xms4g --Xmx4g -. . . -{{< /file >}} - - This configures Elasticsearch with 4GB of allotted RAM. You may also use the `M` letter to specify megabytes, `Xms4096M` in this example. View your current RAM consumption with the `htop` command. If you do not have htop installed, install it with your distribution's package manager. Allocate as much RAM as you can, up to 50% of the max, while leaving enough available for other daemon and system processes. - -1. Restart Elasticsearch for the configurations to take effect: - - systemctl daemon-reload - systemctl restart elasticsearch - -## Configure a Reverse Proxy - -A reverse proxy server allows you to secure the Kibana web interface with SSL and limit access to others. Instructions are provided for NGINX and Apache. The instructions assume you have your webserver configured to host virtual domains. - -### Set up a Reverse Proxy Server to Host Kibana as a Subdomain - -If you have SSL encryption enabled on your domain, follow the instructions in the **HTTPS** section below. If not, follow the instructions included in the **HTTP** section. Although you may skip this section if you wish to access Kibana through its server port, this approach is recommended. - -#### NGINX - -1. Navigate to your NGINX virtual host config directory. Create a new virtual host config file and name it something similar to `example.conf`. Replace `example.com` Add the contents below to this file. If you do not have a domain name available, replace the `server_name` parameter value with your Linode's external IP address: - - **HTTP** - - {{< file "/etc/nginx/conf.d/example.com.conf" >}} -server { - listen 80; - # Remove the line below if you do not have IPv6 enabled. - listen [::]:80; - server_name kibana.exampleIPorDomain; - - location / { - proxy_pass http://exampleIPorDomain:5601; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection 'upgrade'; - proxy_set_header Host $host; - proxy_cache_bypass $http_upgrade; - } - - auth_basic "Restricted Access"; - auth_basic_user_file /etc/nginx/htpasswd.users; -} -{{< /file >}} - - **HTTPS** - - {{< file "/etc/nginx/conf.d/example.com.conf" >}} -server { - listen 80; - # Remove the line below if you do note have IPv6 enabled. - listen [::]:80; - server_name kibana.exampleIPorDomain; - - location / { - proxy_pass http://exampleIPorDomain:5601; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection 'upgrade'; - proxy_set_header Host $host; - proxy_cache_bypass $http_upgrade; - } -} - -server { - listen 443 ssl; - - # Remove the line below if you do not have IPv6 enabled. - listen [::]:443 ssl; - server_name kibana.exampleIPorDomain; - - location / { - proxy_pass http://exampleIPorDomain:5601; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection 'upgrade'; - proxy_set_header Host $host; - proxy_cache_bypass $http_upgrade; - } - - ssl_certificate /path/to/ssl/certificate.crt; - ssl_certificate_key /path/to/ssl/certificate.key; - - auth_basic "Restricted Access"; - auth_basic_user_file /etc/nginx/.htpasswd; -} -{{< /file >}} - -1. Install `httpd-tools` if it is not already installed on your Linode: - - yum install httpd-tools - -1. Secure your Kibana site with a login page. Create a **.htpasswd** file first if you do not have one: - - touch /etc/nginx/.htpasswd - htpasswd -c /etc/nginx/.htpasswd YourNewUsername - chmod 644 /etc/nginx/.htpasswd - -1. Restart the NGINX server to load the new configuration: - - systemctl restart nginx - -### Apache - -1. In order for Apache to function as a reverse proxy, *mod_proxy* must be installed. Check that the following modules are enabled by running the `httpd -M` command: - - httpd -M - - - `proxy_module` - - `lbmethod_byrequests_module` - - `proxy_balancer_module` - - `proxy_http_module` - -1. Enable the necessary mods in Apache. Open `00-proxy.conf` and verify that the lines below are included: - - {{< file "/etc/httpd/conf.modules.d/00-proxy.conf" >}} -. . . -LoadModule proxy_module modules/mod_proxy.so -LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so -LoadModule proxy_balancer_module modules/mod_proxy_balancer.so -LoadModule proxy_http_module modules/mod_proxy_http.so -. . . -{{< /file >}} - -1. Create a new virtual config file for the Kibana site. Add the contents below to this file. If you do not have a domain name available, replace the `server_name` parameter value with your Linode's public IP address. Replace `kibana.exampleIPorDomain` and `http://exampleIPorDomain` with your specific values: - - **HTTP** - - {{< file "/etc/httpd/sites-available/example.com.conf" >}} - - ServerName kibana.exampleIPorDomain - ProxyPreserveHost On - - ProxyPass / http://exampleIPorDomain:5601 - ProxyPassReverse / http://exampleIPorDomain:5601 - - - AuthType Basic - AuthName "Restricted Content" - AuthUserFile /etc/apache2/.htpasswd - Require valid-user - - -{{< /file >}} - - **HTTPS** - - {{< file "/etc/httpd/sites-available/example.com.conf" >}} - - ServerName kibana.exampleIPorDomain - ProxyPreserveHost On - - ProxyPass / http://exampleIPorDomain:5601 - ProxyPassReverse / http://exampleIPorDomain:5601 - - - AuthType Basic - AuthName "Restricted Content" - AuthUserFile /etc/apache2/.htpasswd - Require valid-user - - - - - AuthType Basic - AuthName "Restricted Content" - AuthUserFile /etc/apache2/.htpasswd - Require valid-user - - -{{< /file >}} - -1. Secure your Kibana site with a login page. Create a **.htpasswd** file first if you do not have one: - - touch /etc/apache2/htpasswd.users - htpasswd -c /etc/apache2/.htpasswd.users YourNewUsername - chmod 644 /etc/apache2/.htpasswd.users - -1. Restart Apache: - - systemctl restart httpd - -### Add the Kibana Subdomain to the DNS Manager - -The new Kibana subdomain will need to be configured in the Linode DNS Manager. - -1. Login to the Linode Manager and select **Domains**. Click on your domain's corresponding ellipses and select **Edit DNS Records**. Add a new A/AAA record for the subdomain. Refer to the table below for the field values. - - | Field | Value | - | :-------------: | :--------------------------------------------------: | - | Hostname | Enter your subdomain name here - ex. kibana | - | IP Address | Set this value to your Linode's external IP address. | - | TTL | Set this to 5 minutes. | - -1. Click **Save Changes**. - -## Open the Kibana Port - -Kibana's default access port, `5601`, must be opened for TCP traffic. Instructions are presented below for FirewallD, iptables, and UFW. - -**FirewallD** - - firewall-cmd --add-port=5601/tcp --permanent - firewall-cmd --reload - -1. Set SELinux to allow HTTP connections: - - setsebool -P httpd_can_network_connect 1 - -**iptables** - - iptables -A INPUT -p tcp --dport 5601 -m comment --comment "Kibana port" -j ACCEPT -{{< note >}} -To avoid losing iptables rules after a server reboot, save your rules to a file using `iptables-save`. -{{< /note >}} - -**UFW** - - ufw allow 5601/tcp comment "Kibana port" - -{{% content "cloud-firewall-shortguide" %}} - -## Connect the Elastic Stack with the Wazuh API - -Now you are ready to access the API and begin making use of your OSSEC Elastic Stack. - -1. The Wazuh API requires users to provide credentials in order to login. Navigate to `/var/ossec/api/configuration/auth`. Replace `NewUserName` with whatever user name you choose. Set a password following the system prompts: - - node htpasswd -c user NewUserName - -1. Restart the Wazuh API: - - systemctl restart wazuh-api - -1. Check the status of all daemon components and verify that they are running: - - systemctl -l status wazuh-api - systemctl -l status wazuh-manager - systemctl -l status elasticsearch - systemctl -l status logstash - systemctl -l status kibana - systemctl -l status nginx - - {{< note respectIndent=false >}} -If the Wazuh Manager fails to start and you determine the cause to be one of the OSSEC rules or decoders, disable that specific rule/decoder for now. Find the rules and decoders in the `/var/ossec/ruleset` directory. To disable, rename the file to any other file extension. -{{< /note >}} - -1. In a web browser, navigate to the Kibana homepage. If you created a subdomain for Kibana, the URL will be similar to `kibana.exampleIPorDomain`. You can also reach Kibana by navigating to your server's IP address and specifying port `5601`. Login with the credentials you setup for your Kibana site. - -1. If everything is working correctly, you should have landed on the **Discover** page. Navigate to the **Wazuh** page using the left hand side menu. You will be immediately presented with the API configuration page. Underneath the **ADD NEW API** button, enter the user credentials you created for Wazuh. For URL and Port, enter you URL or IP and `55000`, then click **SAVE**. - -## Where To Go From Here - -Your OSSEC Elastic Stack setup is now complete! At this point, you will want to customize and configure your OSSEC rules to better suit the needs of your environment. The Wazuh API contains pre-configured charts and queries, and more information on how to use them can be found in the official [Wazuh documentation](https://documentation.wazuh.com/current/user-manual/index.html). diff --git a/docs/guides/tools-reference/custom-kernels-distros/custom-compiled-kernel-centos-7/index.md b/docs/guides/tools-reference/custom-kernels-distros/custom-compiled-kernel-centos-7/index.md deleted file mode 100644 index 573bd150783..00000000000 --- a/docs/guides/tools-reference/custom-kernels-distros/custom-compiled-kernel-centos-7/index.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -slug: custom-compiled-kernel-centos-7 -title: 'Custom Compiled Kernel on CentOS 7' -description: 'Instructions for compiling and configuring a custom kernel on your CentOS 7 Linode' -authors: ["Alex Fornuto"] -contributors: ["Alex Fornuto"] -published: 2016-08-02 -modified: 2017-08-01 -keywords: ["compile kernel", "kernel compiling", "custom linux kernel", "custom linode", "centos"] -tags: ["centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -relations: - platform: - key: custom-compiled-kernel - keywords: - - distribution: CentOS 7 -aliases: [] ---- - -![Custom Compiled Kernel on CentOS](custom-compiled-kernel-on-centos-7.png "Custom compiled kernel on CentOS") - -Running a custom-compiled Linux kernel is useful if you need to enable or disable certain kernel features that are not available in Linode-supplied or distribution-supplied kernels. For example, some users desire [SELinux](http://en.wikipedia.org/wiki/Security-Enhanced_Linux) support, which is not enabled in stock Linode kernels, and may not be enabled in some distribution-supplied kernels. - -If you'd rather run a distribution-supplied kernel instead, please follow our guide for [Running a Distribution-Supplied Kernel](https://techdocs.akamai.com/cloud-computing/docs/manage-the-kernel-on-a-compute-instance). - -Prior to these instructions, follow the steps outlined in our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance). Then, log in to your Linode as the `root` user. - -## Prepare the System - -1. Update your package repositories and installed packages, install the development tools required for compiling a kernel, and install the `ncurses` library: - - yum update - yum install -y ncurses-devel make gcc bc openssl-devel grub2 - -2. Since some distributions install a pre-compiled kernel package into the `/boot/` directory along with their development package, avoid confusion later by removing any existing files there. **Warning**, this will delete everything within the `/boot/` directory _without_ asking for confirmation: - - rm -rf /boot/* - -## Compile and Install the Kernel - -### Download Kernel Sources - -1. Download the latest 4.x kernel sources from [kernel.org](http://kernel.org/). A conventional location to download to is `/usr/src/`. - - wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.7.tar.xz - -2. Expand the archived file and change directories: - - tar -xvf linux-4.7.tar.xz - cd linux-4.7 - -### Configure the Kernel - -The kernel must be properly configured to run within the Linode environment. Some required configuration options may include: - - CONFIG_KVM_GUEST=y - CONFIG_VIRTIO_PCI=y - CONFIG_VIRTIO_PCI_LEGACY=y - CONFIG_BLK_DEV_SD - CONFIG_SCSI_VIRTIO=y - CONFIG_VIRTIO_NET=y - CONFIG_SERIAL_8250=y - CONFIG_SERIAL_8250_CONSOLE=y - -We recommend that you start with a kernel configuration (config) from a running Linode kernel. All Linode kernels expose their configuration via `/proc/config.gz`. For example: - - zcat /proc/config.gz > .config - make oldconfig - -`make oldconfig` prompts the user to answer any new configuration options not present in the old configuration file. - -Changes to the kernel's configuration can be made with the `menuconfig` command. Enable any additional options, making sure to leave filesystem support (likely ext3 or ext4) compiled into the kernel (*not* configured as a module). For example, to enable SELinux support, check the option "Security options --\> NSA SELinux Support" in the configuration interface. - - make menuconfig - -Once your configuration options are set, exit the configuration interface and answer "y" for yes when asked whether you would like to save the new kernel configuration. - -### Compile the Kernel - -1. Compile and install the kernel and modules: - - make bzImage - make modules - make - make install - make modules_install - - {{< note >}} - If you're using a Linode with multiple cores, you can use the `j` option to spawn multiple simultaneous jobs to increase speed. For example: - - make -j2 bzImage - {{< /note >}} - -2. Give the kernel a more descriptive name. Modify `vmlinuz-4.7-custom` in the command below to reflect the kernel version you've just compiled: - - mv /boot/vmlinuz /boot/vmlinuz-4.7-custom - -3. Create an `initrd` file. Again, adjust the filename to match the current kernel version: - - mkinitrd /boot/initrd-4.7-custom.img /boot/vmlinuz-4.7-custom - -4. Edit `/etc/default/grub` and add or change the following variables to match. There will be other variables in this file, but we are only concerned with those listed below: - - {{< file "/etc/default/grub" >}} -GRUB_TIMEOUT=10 -GRUB_DISABLE_LINUX_UUID=true -GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0,19200n8" -GRUB_SERIAL_COMMAND="serial --speed=19200 --unit=0 --word=8 --parity=no --stop=1" -GRUB_TERMINAL="serial console" - -{{< /file >}} - - - Comment or remove any lines starting with `GRUB_HIDDEN`. - -5. Make the `grub` directory and build your GRUB configuration file: - - mkdir /boot/grub - grub2-mkconfig -o /boot/grub/grub.cfg - -## Configure the Linode - -1. In the Linode Dashboard, click **Edit** next to your Configuration Profile (usually named after the version of Linux installed). - -2. Under **Boot Settings**, click on the **Kernel** drop-down menu, and select **GRUB2**: - - ![The GRUB2 Option.](custom-kernel-grub2.png) - -3. Click **Save Changes**. You can now reboot the Linode. We suggest opening a [LISH](https://techdocs.akamai.com/cloud-computing/docs/access-your-system-console-using-lish) or [GLISH](https://techdocs.akamai.com/cloud-computing/docs/access-your-desktop-environment-using-glish) session first, so you can monitor the boot process and troubleshoot if necessary. - - -Note that if you install an updated kernel, you need to create a new `initrd` file, and update GRUB. - -Congratulations, you've booted your Linode using a custom-compiled kernel! - -{{< note >}} -You may need to run `cp /boot/grub/unicode.pf2 /boot/grub/fonts/` for the boot menu to properly display in GLISH. Your Linode will still boot, assuming there are no configuration issues, without this command. -{{< /note >}} - diff --git a/docs/guides/tools-reference/tools/duf-command-on-linux-installation/index.md b/docs/guides/tools-reference/tools/duf-command-on-linux-installation/index.md deleted file mode 100644 index 89f4a9a72ce..00000000000 --- a/docs/guides/tools-reference/tools/duf-command-on-linux-installation/index.md +++ /dev/null @@ -1,188 +0,0 @@ ---- -slug: duf-command-on-linux-installation -title: "Install and Use the duf Command on Linux" -title_meta: "How to Install and Use the duf Command on Linux" -description: "Learn how to use the duf utility for checking disk usage and free space on your Linux system. With duf, you get a modern and more user-friendly alternative to both du and df." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2021-11-11 -keywords: ['duf linux','du linux','df linux', 'linux disk usage'] -tags: ['linux'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -Duf is a command-line tool for viewing your system's disk usage and free space. It combines into one place the information you might otherwise get from the `du` and `df` commands and presents that information in a clean and modern interface. In this guide, you learn more about duf, how it compares to the `du` and `df` commands, and how to install it on your Linux system. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -{{< note >}} -The steps in this guide are written for non-root users. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Linux Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## What is duf? - -[duf](https://github.com/muesli/duf) gives you a single tool for checking disk usage and free space. By default, Linux systems provide the du and df tools for viewing disk usage and space, respectively, from the command line. Duf presents the same information as du and df and renders it in a modern and easy-to-read command-line display. - -Where `du` and `df` give you limited control over how information is shown, duf gives you options for sorting, filtering, and otherwise altering the display. You can learn more about `du` and `df` in our [How to Check and Clean a Linux System's Disk Space](/cloud/guides/check-and-clean-linux-disk-space) guide. - -In the following sections, you can see some examples of how duf compares to du and df. Specifically, take a look at the [How to Use duf](/cloud/guides/duf-command-on-linux-installation#how-to-use-duf) section below to see side-by-side comparisons. - -## How to Install duf - -1. Visit the duf [releases page](https://github.com/muesli/duf/releases), find the latest release, and identify the package file appropriate for your machine. Then, copy the URL for the file you identify. - - To do this, you need to know your system's CPU architecture, which you can get via the command below: - - lscpu | grep Architecture - - - For `x86_64` (like in the example output below), use a package ending in `linux_amd64` from the duf releases page. - - For `i386` or `i686`, use a package ending in `linux_386` from the duf releases page. - - Otherwise, you should generally be able to find a package ending in `linux_` followed by your system's listed architecture. - - {{< output >}} -Architecture: x86_64 - {{< /output >}} - - Use the corresponding package file ending in `.deb` if you are on a **Debian** or **Ubuntu** distribution or a `.rpm` file extension if you are on an RHEL distribution like **AlmaLinux**, **CentOS**, or **Fedora**. - - So, for example, on a Debian system with an **x86_64** (AMD64) architecture, use the `duf_0.6.2_linux_amd64.deb` file (assuming **0.6.2** is the latest release). - -1. Download the package file using the command below. Replace the URL with the one you copied in the step above. - - curl -LO https://github.com/muesli/duf/releases/download/v0.6.2/duf_0.6.2_linux_amd64.deb - -1. Install duf from the downloaded package. You can use one of the following methods, depending on your Linux distribution. Be sure to replace the example filename with the actual filename of the package file you downloaded. - - - On **Debian** and **Ubuntu** distributions, use the following command: - - sudo dpkg -i duf_0.6.2_linux_amd64.deb - - - On **AlmaLinux**, **CentOS**, and **Fedora**, use the following command: - - sudo rpm -i duf_0.6.2_linux_amd64.rpm - -1. Verify your installation by checking the installed version of duf. - - duf --version - - {{< output >}} -duf 0.6.2 (d1d2865) - {{< /output >}} - -## How to Use duf - -For basic usage, you can just run the `duf` command alone, and you should see an output similar to the following: - -![Basic duf usage](duf-basic.png) - -This provides roughly the equivalent of the `sudo df -h` command, as you can see from the output below: - -{{< output >}} -Filesystem Size Used Avail Use% Mounted on -udev 3.9G 0 3.9G 0% /dev -tmpfs 798M 8.4M 790M 2% /run -/dev/sda 158G 1.1G 149G 1% / -tmpfs 3.9G 0 3.9G 0% /dev/shm -tmpfs 5.0M 0 5.0M 0% /run/lock -tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup -tmpfs 798M 0 798M 0% /run/user/1001 -{{< /output >}} - -You can use a command like `sudo du / -sh` to get local drive usage alone. - -{{< output >}} -1.1G / -{{< /output >}} - -Notice that, compared to the `du` and `df` commands, `duf` makes the information much easier to read, separating information by device type and clearly demarcating fields. - -You can go even deeper with duf, listing information for pseudo, duplicate, and inaccessible devices as well. - - duf --all - -![Listing all devices in duf](duf-all-devices.png) - -Duf gives an array of additional options to let you control what information you see. You can see the examples of these options presented in the following sections. - -### Limit Results by Path - -You can limit your results by passing one or more paths to the `duf` command, separated by spaces. For each path, `duf` includes an entry for the relevant device, and all non-matching devices are excluded from the output. - -The example below gives `duf` three paths. Notice that the output has three entries, each corresponding to a device for one of the input paths. - - duf /dev /etc/opt /var/log - -![Devices corresponding to given paths in duf](duf-paths.png) - -### Sorting and Arranging - -Duf allows you to define how its displayed tables should be sorted, via the `--sort` option. You can sort by `avail`, `filesystem`, `inodes`, `inodes_avail`, `inodes_usage`, `inodes_used`, `mountpoint`, `size`, `type`, `usage`, or `used`. - -Below is an example sorting the tables by `used`: - - duf --sort used - -![Sorting def by used](duf-sort-used.png) - -Similarly, duf allows you to define what columns you want to show in each table, by way of the `--output` option. This option accepts the same set of columns used for sorting (above), and you can list multiple columns as a comma-separated list (no spaces). - -Below is an example that produces tables with device size, available space, used space, and usage percentage. - - duf --output size,avail,used,usage - -![Defining columns for duf tables](duf-columns-size.png) - -### Filtering - -Duf gives you two options for filtering out devices. - -- You can filter by display table using the `--only` and `--hide` options. Duf displays devices in one of six tables, based on the kinds of devices: `local`, `network`, `fuse`, `special`, `loops`, and `binds`. - - So, to show only devices included in the `local` and `network` tables, use the following command: - - duf --only local, network - - Similarly, to exclude devices included in the `fuse`, `loops`, and `binds` tables, use the following command: - - duf --hide fuse,loops,binds - -- You can filter by devices' file systems using the `--only-fs` and `--hide-fs` options. - - To show only devices with `tmpfs` file systems, for instance, use the following command: - - duf --only-fs tmpfs - -### Export to JSON - -Duf also supports exporting your results in a JSON format. This export option allows the output to be easily used in custom applications. - - duf --json - -{{< output >}} -[ - { - "device": "sysfs", - "device_type": "special", - "mount_point": "/sys", - "fs_type": "sysfs", - "type": "sysfs", - "opts": "rw,nosuid,nodev,noexec,relatime", - "total": 0, - "free": 0, - "used": 0, - "inodes": 0, - "inodes_free": 0, - "inodes_used": 0, - "blocks": 0, - "block_size": 4096 - }, - [...] - {{< /output >}} - -## Conclusion - -Duf makes an easy and capable everyday replacement for du and df, with its quick and clear presentation and much wider range of features. You can learn more about duf, and a few more options it offers, on its official [GitHub page](https://github.com/muesli/duf). diff --git a/docs/guides/tools-reference/tools/faster-file-navigation-with-autojump/index.md b/docs/guides/tools-reference/tools/faster-file-navigation-with-autojump/index.md deleted file mode 100644 index 4b0c702ec52..00000000000 --- a/docs/guides/tools-reference/tools/faster-file-navigation-with-autojump/index.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -slug: faster-file-navigation-with-autojump -title: Faster File Navigation with autojump -description: 'autojump is a cd command that learns - jump to your most frequently visited directories.' -og_description: "Navigate files in the command line more efficiently with autojump, a tool that jumps to your most frequently accessed directories." -authors: ["Sam Foo"] -contributors: ["Sam Foo"] -published: 2017-12-29 -keywords: ["autojump", "python", "command-line"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[autojump GitHub](https://github.com/wting/autojump)' - - '[Autojump Is a Faster Way to Browse Your Filesystem – Lifehacker](https://lifehacker.com/5583546/autojump-is-a-faster-way-to-browse-your-filesystem)' -tags: ["linux"] -aliases: [] ---- - -## What is autojump? - -`autojump` is a command line utility similar to `cd`. It helps speeds up file navigation by maintaining a history of directories that have been previously navigated by the user. If there are directories with the same name, `autojump` maintains a weighted history to favor the most frequently accessed directory. - -## Install autojump - -This guide will cover installation on Linux and MacOS. Support for Windows [is limited](https://github.com/wting/autojump#known-issues). - -### Debian/Ubuntu - -1. Install the `autojump` package: - - sudo apt install autojump - -2. On Debian-based distros, manual activation is required. Add the following line to `~/.bashrc`(for Bash) or `~/.zshrc` (if you use zsh): - - {{< file "~/.bashrc" >}} -. /usr/share/autojump/autojump.sh -{{< /file >}} - -More information is available in the README: - - cat /usr/share/doc/autojump/README.Debian - -### CentOS - - yum install autojump - -For shell specific installation, use `autojump-zsh` for zsh and `autojump-fish` for fish. - -### MacOS - -The recommended installation method is to use [Homebrew](https://brew.sh/): - - brew install autojump - -Be sure to follow the instructions and add the line from installation into the corresponding configuration file. If using `oh-my-zsh`, add `autojump` as a plugin. - - {{< output >}} -Add the following line to your ~/.bash_profile or ~/.zshrc file (and remember -to source the file to update your current session): - [[ -s `brew --prefix`/etc/autojump.sh ]] && . `brew --prefix`/etc/autojump.sh -{{< /output >}} - -### Install autojump via Python - -1. Ensure that git is installed, navigate to the directory where autojump should be installed, then clone the autojump repo: - - git clone git://github.com/joelthelion/autojump.git - -2. Run the install script: - - cd autojump/ - ./install.py - - {{< note respectIndent=false >}} -autojump supports Python 2.6+, with the exception of 3.2. -{{< /note >}} - -## How to Use autojump - -Both `autojump` and `j` are equivalent for most purposes. `j` is preferred for convenience. - -1. Before jumping to any directory, check the weights of the installation: - - j -s - - Since no directories have been visited since installation, the total weight is 0. - - {{< output >}} -________________________________________ - -0: total weight -0: number of entries -0.00: current directory weight - -data: /Users/linode/Library/autojump/autojump.txt -{{< /output >}} - -2. Create an example directory and child. Visit each directory then navigate back to home. - - mkdir -p foo/bar/ - mkdir -p fuu/bar/ - cd foo/ - cd bar/ - cd ~ - cd fuu/ - cd bar/ - cd ~ - -3. Run `j -s` again. The new weights should be reflected in the results: - - {{< output >}} -10.0: /Users/linode/foo -10.0: /Users/linode/foo/bar -10.0: /Users/linode/fuu/bar -10.0: /Users/linode/fuu -________________________________________ - -40: total weight -4: number of entries -0.00: current directory weight - -data: /Users/linode/Library/autojump/autojump.txt -{{< /output >}} - -A more comprehensive description of the arguments can be found with: - - j --help - -### Jump to a Directory - -Jump to a directory: - - j bar - -### Jump to a Child Directory - -Jumping to a child with `c` is supported: - - jc foo - -### Jump with Multiple Arguments - -Multiple arguments can be used with partial names of the full path. - - j fu bar - -### Open Using File Manager - -The `o` command opens the file manager and can also be used in conjunction with `c`. - - jco fuu - -### Purge Deleted Directories from autojump - -When a directory is deleted, its weights remain in autojump's records. You should regularly purge these weights to prevent autojump from navigating to nonexistent directories. - -1. Navigate to the home directory and delete the `foo/` directory: - - cd ~ - rm -rf foo/ - -2. Purge the deleted directory from autojump: - - j --purge - -## Common Issues - -* `autojump` can only be used to jump to directories that have been visited after installation. If you attempt to jump to a directory not yet visited, `autojump` will return `.` - - Visit the directory before attempting to jump. - -* When using `oh-my-zsh`, opening a new Z shell causes the following error: - - {{< output >}} -/Users/linode/.rvm/scripts/initialize:48: __rvm_cleanse_variables: function definition file not found -/Users/linode/.rvm/scripts/initialize:50: command not found: rvm_error -{{< /output >}} - - Make sure `autojump` is added as a plugin in `.zshrc` then remove all `zcomp*` files. - - rm ~/.zcomp* diff --git a/docs/guides/tools-reference/tools/finding-files-with-fd-command/index.md b/docs/guides/tools-reference/tools/finding-files-with-fd-command/index.md deleted file mode 100644 index 9a7e0ccaee7..00000000000 --- a/docs/guides/tools-reference/tools/finding-files-with-fd-command/index.md +++ /dev/null @@ -1,246 +0,0 @@ ---- -slug: finding-files-with-fd-command -title: "Find Files With the fd Command" -title_meta: "How to Find Files With the fd Command" -description: 'This guide shows you how to install and use the fd tool on a Linux system. fd is a user-friendly alternative to the find command.' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2021-11-05 -keywords: ['fd linux command','find command linux','alternative to find','linux fd'] -tags: ['linux'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[fd GitHub page](https://github.com/sharkdp/fd)' ---- - -Linux users typically rely upon the built-in [`find` command](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html) to locate files and directories on their system. It provides a large number of powerful and useful options, but its default behavior often fails to provide intuitive results. The free open-source [*fd*](https://github.com/sharkdp/fd) utility provides a user-friendly alternative to the `find` command. It is more efficient and easier to use, with default behavior that is better suited for the majority of searches. This guide compares the `find` and `fd` commands and explains how to use and install the `fd` command on Linux systems. - -## An Introduction to fd - -As with the `find` command, the `fd` command allows users to find entries in their filesystem. The basic syntax for the command is `fd `. Rather than being a complete replacement for find, fd focuses on simplicity, performance, and ease of use. - -The `fd` command does not support every option that the `find` command does. However, it has a more intuitive syntax with sensible defaults, so users do not usually have to append additional options. For instance, a search in `fd` is only case sensitive if a capital letter is included in the search term. Otherwise, the search is case insensitive. - -According to rigorous benchmark testing on Linux systems, fd generates results much more quickly than find does. It often takes only one-tenth as long to execute the same search and return the same results. fd achieves better performance due to its parallel tree traversal along with improved regex and ignore algorithms. - -## Key Differences Between fd vs find - -Because the `fd` command has been optimized for everyday use, it behaves differently than `find` in certain situations. Here are some of the main differences between the two commands: - -- By default, fd searches are based on directory name or filename. The `-name` or `-iname` options are not required, as they are often used when using the `find` command. -- Support for regular expressions is tightly integrated into fd. -- fd uses color output to highlight file types similar to the `ls` command. -- fd supports parallel command execution. -- Searches in fd are usually case-insensitive, but become case sensitive if a capital letter is included in the search term. The `find` command uses the `-iname` option to control this behavior. -- Default fd searches ignore hidden files and directories, as well as any patterns listed in the `.gitignore` file. - -## Install fd - -The `fd` command is available as part of the default packages on most Linux distributions. Instructions for the most common systems are as follows. - -### Ubuntu / Debian - -For release 19.04 and higher of Ubuntu, the fd application is available through APT. Ubuntu already has a package named `fd`, so the `fd` binary is referred to as `fdfind`. An alias must be added to use the `fd` command. - -{{< note >}} -For information on installing fd on earlier releases of Ubuntu, see [*the fd GitHub page*](https://github.com/sharkdp/fd#installation). -{{< /note >}} - -1. Install fd using APT. - - {{< note respectIndent=false >}} -Use `apt-get` in place of `apt` on Debian distributions. - {{< /note >}} - - sudo apt install fd-find - -1. (**Optional**) Create an alias for `fd` that refers to `fdfind`. To start using the `fd` command immediately, use the `alias` command. - - alias fd=fdfind - -1. (**Optional**) Add this entry to the `bashrc` file to make the alias permanent. - - {{< file "~/.bashrc" >}} -alias fd=fdfind - {{< /file >}} - -### RHEL Derivatives - -On Red Hat-based distributions, fd can be installed using DNF. There is no requirement to create an alias. - - dnf install fd-find - -{{< output >}} -... -Installed: - fd-find-8.1.1-3.fc33.x86_64 - -Complete! -{{< /output >}} - -### Arch - -On the Arch Linux distribution, the fd package can be installed from the official "pacman" repository. - - pacman -S fd - -### Alpine - -On the Alpine distribution, the fd package can be installed using apk. - - apk add fd -{{< note >}} -The `fd` command can also be installed on macOS using Homebrew. Use the command `brew install fd`. For instructions on downloading, installing, and using Homebrew, see the [Homebrew webpage](https://brew.sh/). -{{< /note >}} - -## How to Use fd - -The `fd` command is very useful on its own but can be made even more useful by adding regular expressions and command options. - -To see all of the available flags and options, along with usage instructions, use `fd -h`. The `fd --help` command provides even more detail about each option. - - fd -h - -{{< output >}} -fd 7.4.0 - -USAGE: - fd [FLAGS/OPTIONS] [] [...] - -FLAGS: - -H, --hidden Search hidden files and directories -... - -V, --version Prints version information - -OPTIONS: - -d, --max-depth Set maximum search depth (default: none) -... - -S, --size ... Limit results based on the size of files. - --changed-within Filter by file modification time (newer than) - --changed-before Filter by file modification time (older than) - -ARGS: - the search pattern: a regular expression unless '--glob' is used (optional) - ... the root directory for the filesystem search (optional) - -Note: `fd -h` prints a short overview while `fd --help` gives all details. -{{< /output >}} - -### Basic Search Functionality - -If fd is used without any arguments, it recursively displays all files and directories in the current working directory. This is very similar to the behavior of the `ls -r` command. However, `fd` is typically used with a parameter specifying the search pattern. - -Use `fd ` to find all entries in the current directory that match the search pattern. An entry is considered to be a match if its name contains the search pattern. The command searches all subdirectories recursively, so it also displays any matching entries in any directory that has the current directory in its path. - - fd backup - -{{< output >}} -accounts/payroll/backup_file.sql -accounts/payroll/backup_files.sql -backup -backup/accounts/payroll/backup_file.sql -backup/accounts/payroll/backup_files.sql -mysqlbackup -mysqlbackup/backup_file.sql -mysqlbackup/backup_files.sql -wpbackup -{{< /output >}} - -To search in a specific directory other than the current one, provide the full path of the directory as the second argument. The command pattern for this type of search is `fd `. - - fd backup /usr - -{{< output >}} -/usr/lib/mysql/plugin/component_mysqlbackup.so -... -/usr/src/linux-headers-5.4.0-86-generic/include/config/net/team/mode/activebackup.h -{{< /output >}} - -To list all files in a specific directory, use the wild card symbol `.` as the search pattern. - - fd . ~/wpbackup/public_html/wp-content/themes/twentytwentyone - -### Advanced Search Features - -The `fd` command also allows searches based on regular expressions, file extensions, exact file names, and hidden files. - -Technically, every fd search uses regular expressions. However, the search pattern can be specified in a regex format. The following search looks for entries starting with an `m` and containing the substring `back` at any other position. For more information on the regular expression syntax, consult the [regex documentation](https://docs.rs/regex/1.0.0/regex/). - - fd '^m.*back.*$' - -{{< output >}} -mysqlbackup -{{< /output >}} - -The `fd` command can be used with the `-e` option to find files with a particular extension. In the example below, the command finds all SQL files. - - fd -e sql - -{{< output >}} -accounts/payroll/backup_file.sql -... -mysqlbackup/customer_file.sql -{{< /output >}} - -Typically, fd works in regexp mode and parses the search term as a regular expression. However, adding the `-g` option forces `fd` to perform a glob-based search. This causes it to only display entries that exactly match the search term. In the following search, the `backup` directory matches, but `wpbackup` does not perfectly match and is not listed. - - fd -g backup - -{{< output >}} -backup -{{< /output >}} - -Without any options, `fd` skips over hidden files and directories. These are entries that have names beginning with the `.` character. However, adding the `-H` option causes `fd` to include these entries in its search. - - fd -H bash - -{{< output >}} -.bash_history -.bash_logout -.bashrc -{{< /output >}} - -The `-x` option allows the results to be piped to another command. This is referred to as *command execution*. The format of the command is `fd -x `. For instance, `fd -e txt -x vim` opens each file that matches the search criteria in Vim. To launch the command only once with the list of the files as a string of arguments, use `-X` instead. The `{}` token represents a placeholder for the filename in the target command. This allows for the execution of more complex commands. Consult the [fd documentation](https://github.com/sharkdp/fd) for a full list of all placeholders. - -In the following example, a backup copy is made of each `txt` file found by `fd`. The new file has the same name as the old file with `.bak` appended to the end. The `-x` option invokes the command for each matching entry. - - fd -e txt -x cp {} {}.bak - -For a full list of options, use `fd --help` or consult the [fd GitHub page](https://github.com/sharkdp/fd). Here are a few more options that might be useful: - -- To search for a match on the full path of the file, as opposed to only the filename, use `fd -p`. -- The `-I` option includes both hidden files and those that match a pattern in the `.gitignore` directory. -- `-E ` excludes all entries matching the excluded string. -- `-s` is used to force `fd` to perform a case-sensitive search. -- The `-t ` option is used to filter entries by entry type. Some common types are `f` for file, `d` for the directory, `l` for symlink, and `x` for executable. -- The `-d` option is used to set the maximum search depth in terms of the number of levels of subdirectories. - -### Using fd With Other Programs - -Because fd supports command execution, users can easily integrate other programs. For example, the output from fd can be piped to the as-tree program and represented in tree format. This is usually more useful than running the Linux `tree` command because `fd` has already pre-processed which files to display. To use the two programs together, run the following command. - - fd backup | as-tree - -{{< output >}} -. -├── accounts/payroll -│ ├── backup_file.sql -│ └── backup_files.sql -├── backup/accounts/payroll -│ ├── backup_file.sql -│ └── backup_files.sql -├── mysqlbackup -│ ├── backup_file.sql -│ └── backup_files.sql -└── wpbackup -{{< /output >}} - -For information on installing and using as-tree, consult the [*as-tree GitHub page*](https://github.com/jez/as-tree). - -The [fd documentation](https://github.com/sharkdp/fd) also explains how fd can be used in conjunction with Emacs, the fzf fuzzy finder, and the menu builder rofi. - -## Conclusion - -The `fd` command for Linux is an alternative to the built-in `find` command that is more intuitive and easier to use. fd runs more quickly and has intelligent defaults that align with the needs of most users. It can be installed on most Linux distributions and is usually part of the default package. - -`fd` is not able to do everything the `find` command does, but it has a large number of options that allow users to refine their searches. By default, fd ignores hidden files and is case sensitive only when a capital letter is included in the search term. fd uses regular expressions in its searches, and has powerful regexp capabilities. The `fd` command also supports command execution, which allows the search results to serve as input for another program or command. diff --git a/docs/guides/tools-reference/tools/how-to-administer-server-with-cockpit/index.md b/docs/guides/tools-reference/tools/how-to-administer-server-with-cockpit/index.md deleted file mode 100644 index 21fb048ad57..00000000000 --- a/docs/guides/tools-reference/tools/how-to-administer-server-with-cockpit/index.md +++ /dev/null @@ -1,225 +0,0 @@ ---- -slug: how-to-administer-server-with-cockpit -title: "Monitor and Administer a Server with Cockpit" -title_meta: "How to Monitor and Administer a Server with Cockpit" -description: 'This guide explains how to install and configure Cockpit, a system monitoring and administration tool.' -authors: ["Jeff Novotny"] -contributors: ["Jeff Novotny"] -published: 2022-05-04 -keywords: ['Cockpit linux','Linux cockpit','Cockpit server management','Cockpit ubuntu server'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[Cockpit website](https://cockpit-project.org/)' -- '[Cockpit applications page](https://cockpit-project.org/applications.html)' -- '[Cockpit installation instructions](https://cockpit-project.org/running.html)' -- '[Cockpit documentation](https://cockpit-project.org/guide/latest/)' -- '[Webmin](https://www.webmin.com/)' -- '[Netdata](https://www.netdata.cloud/)' -- '[cPanel](https://cpanel.net/)' -relations: - platform: - keywords: - - distribution: Ubuntu 22.04 LTS ---- - -It can be frustrating to manage a Linux server solely from the terminal, so users are always searching for an easier and more intuitive option. One of the more promising applications in this area is [Cockpit](https://cockpit-project.org/). Cockpit allows users to monitor and configure a server through a graphical user interface. This guide explains how to use Cockpit on Linux for server management and provides some background about the application. - -## What is Cockpit? - -Cockpit is a server management tool that is available as a Linux plugin. It allows users to manage Linux systems using a web-based graphical interface and dashboard. Cockpit is designed for all users, both those who are new to Linux and experienced administrators. Cockpit is efficient, lightweight, and easy to use. - -Cockpit provides a wrapper for Linux commands and uses existing system APIs, rendering it more robust against change. There is no requirement to remember the syntax of any command. Many tasks can be accomplished with a single click. Cockpit can co-exist with a web server, but a web server is not required to use Cockpit. - -Here are some of Cockpit's advantages and features: - -- It is a free and open source application. Cockpit for Linux is available under the GNU Lesser General Public License. -- It features an intuitive design. Each aspect of the server is separated into a different screen. -- It allows users to change network settings, manage storage, configure a firewall, browse system logs, upgrade software, oversee user accounts, and enable system services. -- It includes tools for monitoring performance and diagnosing network problems. It provides metrics for CPU utilization, memory use, network activity, and storage availability. -- It incorporates a built-in terminal to allow users to toggle between the command line and GUI. -- It can manage virtual machines, and download and run containers. -- Cockpit allows users to quickly switch between multiple servers. -- It is efficient and does not use resources when not active. -- It provides multi-user support, allowing a team to manage a server. -- It is compatible with existing tools like Ansible. Users can switch back and forth between tools as required. -- Cockpit supports many optional and third-party extensions. These applications extend Cockpit's capabilities even further. A full list can be found on the [Cockpit applications page](https://cockpit-project.org/applications.html). -- Cockpit includes a "starter kit" for creating custom and user-defined modules. - -## Cockpit vs Other Server Management Tools - -Cockpit is one of many available server management tools. Some of the other options include [Webmin](https://www.webmin.com/), [Netdata](https://www.netdata.cloud/), and [cPanel](https://cpanel.net/). Each of these tools can be contrasted with Cockpit in terms of its own relative strengths and weaknesses. - -- **Netdata**: Netdata is a free and open source application for managing both systems and applications. It is known for its high-quality visualization and interactive tools. Netdata is considered a strong alternative to Cockpit, but it is more focused on real-time monitoring. -- **Webmin**: Webmin is also a web-based system configuration tool. It can manage both Windows and Linux/UNIX systems. In addition to managing servers, Webmin can also modify applications and LAMP stack components. However, Cockpit is considered to have a more intuitive user interface. -- **cPanel**: cPanel is best known as a web host manager. However, it can also manage many server configuration tasks. cPanel is very well known because many web hosting services provide it to their users at no extra cost. However, it does not focus on server management like Cockpit does. cPanel is a paid service, not a free or open source alternative. - -Other tools including Ansible, Strapi, and Portainer serve complementary roles. They provide container management and IT automation tools, or serve primarily as *content management systems* (CMS). They can be used in conjunction with Cockpit to provide stronger capabilities in their area of expertise. In particular, Ansible and Cockpit are often used together, with Ansible providing an automation framework. Users can choose the best application for their current task. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you are not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## How to Install Cockpit on Ubuntu 22.04 LTS - -These instructions are designed for Ubuntu 22.04 LTS, but are generally valid for Ubuntu releases 20.xx and 18.xx as well. For other Linux distributions, refer to the [Cockpit installation instructions](https://cockpit-project.org/running.html). - -### Installing Cockpit - -To install Cockpit on Linux, follow these steps. - -1. Ensure software packages for the system are up to date. If prompted, press **Y** to continue and restart the system if necessary. - - ```command - sudo apt update && sudo apt upgrade - ``` - -2. Use `apt` to install Cockpit and press **Y** to continue. - - ```command - sudo apt install cockpit - ``` - -3. The Ubuntu server does not start Cockpit automatically. Launch the Cockpit service using the `systemctl` service. Both the `cockpit` and `cockpit.socket` processes must be started. - - ```command - sudo systemctl start cockpit cockpit.socket - ``` - -4. Confirm Cockpit is now running using the `systemctl status` command. When done, press **CTRL+C** to close the `systemctl status` output. - - ```command - sudo systemctl status cockpit - ``` - - {{< output >}} -cockpit.service - Cockpit Web Service - Loaded: loaded (/lib/systemd/system/cockpit.service; static) - Active: active (running) since Thu 2022-05-05 15:55:21 UTC; 5s ago -TriggeredBy: ● cockpit.socket - {{< /output >}} - - {{< note respectIndent=false >}} -Cockpit becomes dormant after a period of inactivity. In this case, it might display a status of `inactive (dead)`. Cockpit automatically wakes up when a user accesses it through its web interface. To confirm it is installed correctly, restart it using the previous instruction. - {{< /note >}} - -5. **(Optional)** To automatically activate Cockpit at start up, use the following command. - - ```command - sudo systemctl enable cockpit cockpit.socket - ``` - -6. To allow Cockpit connections through the firewall, allow port `9090` in `ufw`. Ensure `ufw` is configured to allow `OpenSSH` before it is enabled. Enable the firewall after entering all commands and press **Y** to proceed. - - ```command - sudo ufw allow OpenSSH - sudo ufw allow 9090 - sudo ufw enable - ``` - -7. Verify the firewall status. `ufw` should have a status of `active` and port `9090` should be allowed through the firewall. - - ```command - sudo ufw status - ``` - - {{< output >}} -Status: active - -To Action From --- ------ ---- -OpenSSH ALLOW Anywhere -9090 ALLOW Anywhere -OpenSSH (v6) ALLOW Anywhere (v6) -9090 (v6) ALLOW Anywhere (v6) - {{< /output >}} - -8. Cockpit is now ready to use. In a web browser, enter the IP address of the server, the `:` symbol, and port number `9090`. The Cockpit login page is then displayed. - - ```command - https://Ip_address:9090/ - ``` - - {{< note respectIndent=false >}} -If HTTPS is not enabled, the web browser displays a warning and tells the user the connection might not be safe. To bypass this warning, Click **Advanced** and then accept the certificate. The actual instructions vary depending on the browser. - {{< /note >}} - - ![Cockpit Login Page](Cockpit-Login-Screen.png) - -### Configuring Your Server - -Many server components can be configured directly from Cockpit without any Linux commands or access to a terminal console. The [Cockpit documentation](https://cockpit-project.org/guide/latest/) provides complete instructions on how to use the application. For a brief overview of Cockpit's configuration capabilities, follow these introductory steps. - -1. To start configuring a Linux server, first log in to Cockpit. To add the server to Cockpit, enter the user name of the server account and the password. The account must have `sudo` access. Alternatively, log in using the root server account. Although it is not necessary, a new account can be created to manage the Cockpit account. - - ![Cockpit Adding Login Details](Cockpit-Login-Details.png) - -2. Upon first login, you may find a yellow banner across the top of the screen that reads "Web console is running in limited access mode." Click the blue **Turn on administrator access** button and re-enter your user password to enable full access. - - ![Cockpit Limited Access Warning](Cockpit-Limited-Access.png) - -3. After validating the account credentials, Cockpit redirects the browser to the main Cockpit dashboard. The main **Overview** screen serves as the system dashboard. It provides a quick overview of the system. Each subsystem is listed in a menu bar on the left. The overview provides a summary of the basic status and health of the system, along with the CPU and memory usage. - - ![Cockpit Dashboard Overview](Cockpit-Dashboard.png) - -4. From the overview page, several options are available. To add a new account, click on the **Accounts** link in the left menu. This screen displays details about all user accounts associated with the system. - - ![Cockpit Accounts Tab](Cockpit-Accounts.png) - -5. Click on any account to view details about the account. To create an account, select the **Create new account** button. At the next screen, enter the account details. Add a user name and password for the new account. Click **Create** to add the account or **Cancel** to cancel. - - ![Cockpit New Account Screen](Cockpit-Create-Account.png) - -6. Use the Networking window to add bridges, VLANs, and other network components. Click on the **Networking** option in the left-hand menu. This screen displays the rate at which the system is transmitting and receiving data. It also lists the managed and unmanaged interfaces. Cockpit displays the network logs at the bottom of the screen underneath the other widgets. Scroll downwards to view the logs in reverse chronological order. - - ![Cockpit Networks Tab](Cockpit-Networking.png) - -7. To add a bridge or VLAN, use the corresponding button on the right side of the Networking window. For example, choose the **Add VLAN** button to add a VLAN. Then add the details about the new service. A bridge can be added through the **Add bridge** button. - - ![Cockpit VLAN screen](Cockpit-Add-VLAN.png) -{{< note >}} -To add a new server to Cockpit, click the arrow next to the user account details in the upper-left corner. You can either add a new host or select/search for an existing host. -{{< /note >}} - -### How to Use Cockpit to Manage Your Server - -Cockpit also makes it easy for administrators to manage and monitor an Ubuntu server. The following Cockpit server management options are available. - -- **System Monitoring**: On the main **Overview** screen, Cockpit provides a quick snapshot of the CPU and Memory usage. From the **Overview** page, click **View details and history** to view more performance metrics. - - ![Cockpit Performance Metrics screen](Cockpit-Performance-Metrics.png) - -- **Service Management**: Cockpit lets users manage and add new services. To access this page, click **Services** on the left-hand navigation panel. This screen displays a list of all the services. At the very top, Cockpit highlights any services that have failed to launch or have crashed. - - ![Cockpit Services screen](Cockpit-Services.png) - -- **Manage an Individual Service**: To manage a service, click on the service name. This displays a screen with more information about the service and the service logs. Click the button next to the service name to stop/disable or start/enable the service. The three dots next to the service name enables more options, such as "Restart". In the following screenshot, Cockpit displays information about the `mysql` service. - - ![Cockpit MySQL Service screen](Cockpit-Services-MySQL.png) - -- **Upgrade Software**: Cockpit allows users to automatically upgrade their software. Select **Software Updates** from the left-hand menu to see what updates are available. Click **Install all updates** to automatically upgrade the system. - - ![Cockpit Software Updates screen](Cockpit-Software-Upgrade.png) - -- **View Disc Usage**: Click on the **Storage** link in the menu to see how much system storage is being used. This screen lists each file system and allows users to see how often the storage drives are accessed. Configure a RAID device using the blue box in the upper-right corner, or mount an NFS drive by clicking the blue **Install NFS Support** button. Click on the name of a file system to view more information about it. This page also displays logs related to the storage system or the individual drives. - - ![Cockpit Storage screen](Cockpit-Storage.png) - -- **Analyze Logs**: To view all the system logs, click **Logs** from the left-hand menu bar. This screen allows users to filter the logs based on time, priority, process ID, or another search phrase. Click on any log item to learn more about it. - - ![Cockpit Logs screen](Cockpit-Logs.png) - -- **Terminal Access**: Cockpit allows users to toggle between its user interface and a system terminal for command line access. Click **Terminal** in the left-hand sidebar to open a system terminal. The user does not have to log in again. - - ![Cockpit Terminal](Cockpit-Terminal.png) - -## A Summary of the Linux Cockpit Application - -Cockpit is a server management tool for Linux-based systems. It provides an intuitive and easy to use graphical user interface. Cockpit permits multi-user access and can switch between servers. It is very efficient and does not use much memory or CPU when not active. Cockpit can be installed on Ubuntu using the `apt` utility. - -Cockpit allows users to perform Linux system configuration tasks. It provides the ability to add, remove, or delete network services, user accounts, and user services. It also supplies some sophisticated system monitoring tools. Users can see a snapshot of CPU and memory utilization, upgrade software components, view storage consumption, and read the system logs. It can even open a terminal window to the server right within the GUI. For more information about Cockpit, consult the [Cockpit website](https://cockpit-project.org/). \ No newline at end of file diff --git a/docs/guides/tools-reference/tools/how-to-use-gping-on-linux/index.md b/docs/guides/tools-reference/tools/how-to-use-gping-on-linux/index.md deleted file mode 100644 index e1788d93c2d..00000000000 --- a/docs/guides/tools-reference/tools/how-to-use-gping-on-linux/index.md +++ /dev/null @@ -1,143 +0,0 @@ ---- -slug: how-to-use-gping-on-linux -title: "Install and Use gping on Linux" -title_meta: "How to Install and Use gping on Linux" -description: "Learn how to use gping, a Linux tool with the functionality of ping and a modern graphical command-line display." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2021-10-22 -keywords: ['gping linux','ping linux','ping linux alternatives'] -tags: ['linux'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -The gping tool takes the functionality of the ping tool and displays its data on a graph. With gping you can track the response time for hosts, and compare the data side by side. This guide shows you how to install gping and get started using it on your Linux system. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## What is gping? - -[gping](https://github.com/orf/gping) is a Linux tool that provides the same functionality as ping with information displayed on a graph. If you need to measure the response time of a host over time, gping is a tool for the task. With gping, you can ping multiple hosts simultaneously, making it easy to compare response times. You can customize how your gping graph displays information and it is able to graph the execution time of Linux commands. - -## How to Install gping - -This guide provides installation steps for Debian, Ubuntu, AlamaLinux, CentOS, and Fedora. Find the section below that corresponds to your system's distribution to learn how to install gping. - -### Debian and Ubuntu - -On Debian and Ubuntu distributions, use the following steps to install gping. - -1. Install the `gnupg` package, which is used for adding a repository key to APT: - - sudo apt install gnupg - -1. Add the Azlux repository to APT. Replace `buster` in the command below with the codename for your Debian release: - - echo "deb http://packages.azlux.fr/debian/ buster main" | sudo tee /etc/apt/sources.list.d/azlux.list - - On Debian, you can find your release's codename with the following command: - - lsb_release -a | grep Codename - - For example, if you are on a Debian 10 system, the above command returns the following output: - - {{< output >}} -Codename: buster - {{< /output >}} - - On Ubuntu, you can find the Debian codename with the command below: - - cat /etc/debian_version - - If you are on an Ubuntu 20.04 system, the above command returns the following output: - - {{< output >}} -bullseye/sid - {{< /output >}} - -1. Add the Azlux repository key to APT: - - wget -qO - https://azlux.fr/repo.gpg.key | sudo apt-key add - - -1. Update the APT registry, and install `gping`: - - sudo apt update - sudo apt install gping - -### AlmaLinux, CentOS, and Fedora - -On AlmaLinux, CentOS (8 and later), and Fedora, use the following steps to install gping. - -1. Use the COPR plugin to enable the repository for `gping`: - - sudo dnf copr enable atim/gping - -1. Install gping: - - sudo dnf install gping - -## How to Use gping - -To use the gping, issue the `gping` command followed by a hostname or IP address. The example below pings `github.com`: - - gping github.com - -![gping graph for github.com](gping-host.png) - -You can also use gping to graph multiple hosts' ping times simultaneously. The next example pings `github.com` and `gitlab.com`, each graphed in a distinct color for readability: - - gping github.com gitlab.com - -![gping graphs for github.com and gitlab.com](gping-multiple-hosts.png) - -Should you need a specific kind of IP address resolution, gping supports `-4` and `-6` options for resolving addresses to IPv4 and IPv6, respectively. - -gping provides options you can use to fine-tune its graph and also has an option to graph execution time of system commands. The next sections dive deeper into these command options and provide examples that show you how to use them. - -### Adjusting Display Options - -gping gives you control over the following two aspects of its graph: - -- The buffer time, using the `--buffer` option followed by a number of seconds. This determines how much time is shown on the graph screen. You can see in the screenshots above, from the times given below the graphs, that the default buffer time is **30** seconds. - -- The watch interval, using the `--watch-interval` option followed by a number of seconds, including fractions of a second. This controls how frequently `gping` measures ping time. The default is **0.5** seconds. - -The example employs both of these options to alter the display. The command below has `gping` display 60 seconds at a time and has it graph the ping time of `github.com` once every second: - - gping github.com --buffer 60 --watch-interval 1 - -![gping with display timing adjustments](gping-timing.png) - -If you need to simplify how gping renders the graph line, you can use the option `--simple-graphics`. With this option, gping uses dot characters instead of braille to render its graph lines. This can be useful on terminals that do not support braille rendering: - -![gping with a simplified graph line](gping-simplified.png) - -### Graphing Command Execution Time - -As a bonus feature, gping allows you to graph execution time for command-line commands. Using the `--cmd` option, gping lets you provide commands much as you would normally provide hostnames or IP addresses. For example: - - gping --cmd 'ls /' - -![gping graphing execution time for the 'ls /' command](gping-command.png) - -As with hosts, you can pass gping multiple commands when using this option to see the commands' execution times graphed side by side: - - gping --cmd 'ls' 'ls /' - -![gping graphing execution times for the 'ls' 'ls /' commands](gping-multiple-commands.png) - -Since graphing command execution times is a bonus gping feature, it may not perform reliably for all kinds of commands. This is especially the case for commands that are more complicated. - -If you are looking to measure a command's performances, consider using the hyperfine tool. Hyperfine is a tool for benchmarking command-line commands, with an emphasis on clearly rendering side-by-side comparisons. If you are interested, check out our guide on [getting started with hyperfine](/cloud/guides/installing-and-using-hyperfine-on-linux). - -## Conclusion - -You are now ready to start using gping to measure response times. Due to the ping tool's simplicity, gping might not fully replace it, however, gping fills a highly useful niche. If you need to measure a host's response times over some duration, it is useful to have gping installed and ready on your system. diff --git a/docs/guides/tools-reference/tools/tldr-pages-on-linux/index.md b/docs/guides/tools-reference/tools/tldr-pages-on-linux/index.md deleted file mode 100644 index c05f1e8a924..00000000000 --- a/docs/guides/tools-reference/tools/tldr-pages-on-linux/index.md +++ /dev/null @@ -1,277 +0,0 @@ ---- -slug: tldr-pages-on-linux -title: "Use tldr pages on Linux" -title_meta: "How to Use tldr pages on Linux" -description: "This guide shows you how to install tldr pages and use the tldr command-line tool. With tldr pages, you get a more approachable version of traditional man pages, making it easier to get started with new command-line tools." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2021-10-29 -keywords: ['install tldr','tldr pages linux','tldr man pages','tldr linux command'] -tags: ['linux'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -The [tldr pages project](https://tldr.sh/) seeks to improve and simplify the well-known man pages. tldr pages provide information on a wide range of Linux commands and include several options to interact with the information. Compared to man pages, each tldr page aims to be more intuitive and readable. When learning new Linux commands, tldr pages are an excellent resource. - -In this guide, you learn more about the tldr pages project, how to install tldr pages on a Linux system, and how to use the `tldr` command. This guide also provides a comparison between man pages and tldr pages. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -{{< note >}} -The steps in this guide are written for non-root users. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Linux Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## What are tldr pages? - -The [tldr pages project](https://github.com/tldr-pages/tldr) aims to provide a simple alternative to man pages, one that emphasizes readability and practical examples. - -As with man pages, you can use tldr pages to find a description of a command and its available options. But, unlike man pages, tldr pages does not try to provide a comprehensive list of options for each command. Instead, it focuses on each command's most useful options and gives clear and practical examples for each. - -The result is a resource that can make it much easier to learn and start using a new command. A command's man page can be useful if you know the command and want to work out specific actions. However, it can be difficult to engage with man pages if you are less familiar with the command. A command's tldr page, on the other hand, focuses on introducing you to a command. It gives you everything you need to know short of advanced usage. - -## How to Install tldr - -To interact with tldr pages, you can use one of the tldr clients. These give you access to tldr pages from the command line or, in some cases, other platforms. - -Several tldr clients are available, and you can see the full listing on the tldr [GitHub page](https://github.com/tldr-pages/tldr#how-do-i-use-it). - -This guide gives you installation steps for two of the most popular tldr clients, one using **Node.js** and the other using **Python 3**. Take a look at each of the tldr clients in the sections below to learn how to install them on your Linux system. - -### Using Node.js - -1. Install the [node package manager (NPM)](https://www.npmjs.com/). The recommended way to do this is by first installing the Node Version Manager (NVM). - - You can use the series of commands shown below to first install NVM and then use it to install the current version of Node.js. The Node.js installation includes the current NPM release. - - Before proceeding, check the NVM [releases page](https://github.com/nvm-sh/nvm/releases), and replace `v0.38.0` from the command below with the version number of the latest release you find on the releases page. - - curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash - source ~/.bashrc - nvm install node - - You can then verify your NPM installation using the following command: - - npm --version - - {{< output >}} -7.21.0 - {{< /output >}} - -1. Install `tldr` via NPM. Use the `-g` option to install `tldr` as a global NPM package. - - npm install tldr -g - -### Using Python 3 - -1. Make sure you have Python 3 installed. - - On **Debian** and **Ubuntu**, recent releases include Python 3 by default, which you can verify using the `--version` flag. - - python3 --version - - {{< output >}} -Python 3.8.10 - {{< /output >}} - - On **AlmaLinux**, **CentOS** (8 or later), and **Fedora**, use the command below to install Python 3. - - sudo dnf install python3 - -1. Install tldr via Pip 3, the default package installer for Python 3. - - sudo pip3 install tldr - -## How to Use tldr - -You can start using tldr pages by issuing the `tldr` command followed by the name of a command you want to know more about. For example, you can get the tldr page for the `ls` command as shown below: - - tldr ls - -{{< output >}} - ls - - List directory contents. - More information: https://www.gnu.org/software/coreutils/ls. - - - List files one per line: - ls -1 - - - List all files, including hidden files: - ls -a - - - List all files, with trailing `/` added to directory names: - ls -F - - - Long format list (permissions, ownership, size, and modification date) of all files: - ls -la - - - Long format list with size displayed using human-readable units (KiB, MiB, GiB): - ls -lh - - - Long format list sorted by size (descending): - ls -lS - - - Long format list of all files, sorted by modification date (oldest first): - ls -ltr - - - Only list directories: - ls -d */ -{{< /output >}} - -By way of contrast, here is an excerpt from the extensive man page for the `ls` command. - - man ls - -{{< output >}} -LS(1) User Commands LS(1) - -NAME - ls - list directory contents - -SYNOPSIS - ls [OPTION]... [FILE]... - -DESCRIPTION - List information about the FILEs (the current directory by default). Sort entries alphabetically if none of -cftuvSUX nor --sort is specified. - - Mandatory arguments to long options are mandatory for short options too. - - -a, --all - do not ignore entries starting with . - - -A, --almost-all - do not list implied . and .. - - --author - with -l, print the author of each file - - -b, --escape - print C-style escapes for nongraphic characters - - --block-size=SIZE - with -l, scale sizes by SIZE when printing them; e.g., '--block-size=M'; see SIZE format below - - -B, --ignore-backups - do not list implied entries ending with ~ -[...] -{{< /output >}} - -As you can see, the tldr page focuses on giving you clear descriptions and examples of some of the most relevant options for the command. The man page, on the other hand, focuses on a comprehensive listing of options. For this reason, man pages' option descriptions are not always clear and can fall short in helping you find the most useful options. - -Below is another example that goes further in illustrating the contrast between tldr pages and man pages. - - tldr vim - -{{< output >}} - vim - - Vim (Vi IMproved), a command-line text editor, provides several modes for different kinds of text manipulation. - Pressing `i` enters insert mode. `Esc` enters normal mode, which enables the use of Vim commands. - More information: https://www.vim.org. - - - Open a file: - vim path/to/file - - - Open a file at a specified line number: - vim +line_number path/to/file - - - View Vim's help manual: - :help - - - Save and Quit: - :wq - - - Undo the last operation: - u - - - Search for a pattern in the file (press `n`/`N` to go to next/previous match): - /search_pattern - - - Perform a regular expression substitution in the whole file: - :%s/regular_expression/replacement/g - - - Display the line numbers: - :set nu -{{< /output >}} - -And again, the output below displays an excerpt from the man page for comparison. - - man vim - -{{< output >}} -VIM(1) General Commands Manual VIM(1) - -NAME - vim - Vi IMproved, a programmer's text editor - -SYNOPSIS - vim [options] [file ..] - vim [options] - - vim [options] -t tag - vim [options] -q [errorfile] - - ex gex - view - gvim gview vimx evim eview - rvim rview rgvim rgview - -DESCRIPTION - Vim is a text editor that is upwards compatible to Vi. It can be used to edit all kinds of plain text. It is especially useful for editing programs. - - There are a lot of enhancements above Vi: multi level undo, multi windows and buffers, syntax highlighting, command line editing, filename completion, on-line help, - visual selection, etc.. See ":help vi_diff.txt" for a summary of the differences between Vim and Vi. - - While running Vim a lot of help can be obtained from the on-line help system, with the ":help" command. See the ON-LINE HELP section below. - - Most often Vim is started to edit a single file with the command - - vim file - - More generally Vim is started with: - - vim [options] [filelist] - - If the filelist is missing, the editor will start with an empty buffer. Otherwise exactly one out of the following four may be used to choose one or more files to be - edited. - - file .. A list of filenames. The first one will be the current file and read into the buffer. The cursor will be positioned on the first line of the buffer. You - can get to the other files with the ":next" command. To edit a file that starts with a dash, precede the filelist with "--". - - - The file to edit is read from stdin. Commands are read from stderr, which should be a TTY. - - -t {tag} The file to edit and the initial cursor position depends on a "tag", a sort of goto label. {tag} is looked up in the tags file, the associated file becomes - the current file and the associated command is executed. Mostly this is used for C programs, in which case {tag} could be a function name. The effect is - that the file containing that function becomes the current file and the cursor is positioned on the start of the function. See ":help tag-commands". - - -q [errorfile] - Start in quickFix mode. The file [errorfile] is read and the first error is displayed. If [errorfile] is omitted, the filename is obtained from the - 'errorfile' option (defaults to "AztecC.Err" for the Amiga, "errors.err" on other systems). Further errors can be jumped to with the ":cn" command. See - ":help quickfix". -[...] -{{< /output >}} - -Notice that, with the tldr page, there is little reference to command-line arguments. This is because Vim does not frequently use them. Instead, the tldr page focuses on the commands you may want to use within Vim. This is much more helpful when getting started using a tool like Vim. - -By contrast, the man page does not provide any information about commands you can use once you have Vim up and running. Instead, true to its goal, the man page gives you an exhaustive list of command-line options. It even includes the seldom-used alternative command names for starting Vim in different modes. - -### How to Use tldr pages and man Pages Together - -tldr pages work best when used in conjunction with man pages. With a command's tldr page, you get a clear and succinct introduction to the command. Its examples and focus on useful options makes the tldr page a valuable place to start. - -The format used by tldr pages does leave some information out for many commands. For instance, say you see the following `ls` command: - - ls -Alh - -The tldr page informs you that the `-l` and `-h` options give you a long listing format and human-readable file sizes, respectively. But then, what does the `-A` option do? That is where the man page can be useful. - -Typically, this is a good method to keep in mind when using tldr pages. A command's tldr page frequently contains all you need to get started; sometimes, you need nothing more than what the tldr page offers. If you find yourself needing to dig deeper into a command, build off of what you learn on the tldr page with the [wealth of details](https://man7.org/linux/man-pages/man1/man.1.html) you can find on the man page. - -## Conclusion - -You now have the knowledge you need to get started using tldr pages. It might be helpful to go ahead and start checking out the tldr pages for some commands you are already using. That way, you get to know tldr pages better and potentially learn how to get more out of those commands. - -Do you see something missing while looking over a tldr page? Is there a command you notice does not have a tldr page yet? Or, do you just like what tldr pages is doing and want to be a part of it? The tldr pages project maintainers keep an open invitation for contributors. All of the tldr pages are stored on the project's [GitHub repository](https://github.com/tldr-pages/tldr), and the tldr team provides a [guide for contributing](https://github.com/tldr-pages/tldr/blob/main/CONTRIBUTING.md). diff --git a/docs/guides/tools-reference/tools/use-dog-linux-dns-client/index.md b/docs/guides/tools-reference/tools/use-dog-linux-dns-client/index.md deleted file mode 100644 index 044792bf850..00000000000 --- a/docs/guides/tools-reference/tools/use-dog-linux-dns-client/index.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -slug: use-dog-linux-dns-client -title: "Use the Linux dog Command to Look Up DNS Records" -description: "Learn how to use the dog command-line DNS client, a modern and more user-friendly alternative to dig." -authors: ["Nathaniel Stickman"] -contributors: ["Nathaniel Stickman"] -published: 2021-12-23 -keywords: ['dog dns client','dig alternative linux','dig command examples'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' ---- - -`dog` is a command-line DNS client used for looking up DNS records for domain names. It's an alternative to the popular `dig` command. The `dog` command gives you a simpler interface, more readable results, and additional features like DNS over TLS. - -In this guide, learn more about `dog` and how to install and start using it on your Linux system. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## What is dog? - -[`dog`](https://github.com/ogham/dog) is an open-source DNS client for the command line, much like the popular `dig` tool. With `dog`, you get significant improvements to the interface, along with more readable, color-coded results, and the ability to render those results in JSON. `dog` also adds support for DNS over TLS (DoT) and DNS over HTTPS (DoH) protocols, giving you more options for securing your DNS lookups. - -You can learn more about the `dig` command and its features in our guide [Use dig to Perform Manual DNS Queries](/cloud/guides/use-dig-to-perform-manual-dns-queries). - -## How to Install dog - -1. Install `gcc`, `tar`, and the developer package for `libssl` or `openssl`. Choose the command for your particular Linux distribution. - - - On **Debian** and **Ubuntu**, you can do so with: - - sudo apt install build-essential tar libssl-dev pkg-config - - - On **AlmaLinux**, **CentOS**, and **Fedora**, you can use: - - sudo dnf install gcc tar openssl-devel - - {{< note respectIndent=false >}} -You may need to update your system's version of the GNU C library (glibc). -{{< /note >}} - -1. Install [Rust](https://www.rust-lang.org/). You need Rust to compile the `dog` source code: - - curl --proto =https --tlsv1.2 -sSf https://sh.rustup.rs | sh - - When prompted, select `1` for the default installation path. - -1. Either restart your shell session (exiting and logging back in) or run the following command: - - source $HOME/.cargo/env - -1. Navigate to the [releases page](https://github.com/ogham/dog/releases/) for `dog`, identify the latest release, and copy the URL for the `.tar.gz` file. - - {{< note respectIndent=false >}} -To access the `.tar.gz` file, navigate to the [**Tags** section](https://github.com/ogham/dog/tags) of the dog releases page. - {{< /note >}} - -1. Download that file, replacing the URL below with the one you copied: - - curl -LO https://github.com/ogham/dog/archive/refs/tags/v0.1.0.tar.gz - -1. Extract the contents of the `.tar.gz` file, and change into the extracted directory. Replace the filename below with the one for the file you downloaded. Likewise with the directory name, matching the extracted one: - - tar -xvzf v0.1.0.tar.gz - cd dog-0.1.0 - -1. Run the following command to have Cargo compile the binary for `dog`: - - cargo build --release - -1. Copy the resulting binary into your current user's `PATH`: - - sudo cp target/release/dog /usr/local/bin - -1. Verify your installation by checking the installed version of `dog`: - - dog --version - - {{< output >}} -dog ● command-line DNS client -v0.1.0 -https://dns.lookup.dog/ - {{< /output >}} - -## How to Use dog - -`dog` gives you much of the same functionality of `dig`, but pared down to the essential DNS records. This makes `dog`'s results easier to read and more manageable. - -In the section below, you can see how to get started with basic `dog` queries and learn more about its advanced options. If you want to learn more about DNS and its role in managing your servers, refer to the end of this guide for more resources. - -### Basic Queries - -At its simplest, you can start looking up DNS records with `dog` just by giving it a hostname: - - dog github.com - -{{< output >}} -A github.com. 51s 192.0.2.0 -{{< /output >}} - -The output includes the record type (**A**), the domain name, the time until the record is refreshed (51 seconds), and the record's main contents — a host IP address, in this case. The main contents for a record vary depending on the record type, which you can see with the next example. - -`dog` provides color codes to portions of the records it displays. This helps you navigate the information when your response includes several records, like in the image for the next example command below. - -`dog` looks up **A** type records by default, which contain IPv4 addresses. But you can easily add more record types to your `dog` lookup, like this: - - dog github.com A AAAA MX NS TXT - -{{< output >}} - A github.com. 40s 192.0.2.0 -SOA github.com. 58m20s A "dns1.p08.nsone.net." "hostmaster.nsone.net." 1633608682 12h00m00s 2h00m00s 14d0h00m00s 1h00m00s -SOA github.com. 58m20s A "dns1.p08.nsone.net." "hostmaster.nsone.net." 1633608682 12h00m00s 2h00m00s 14d0h00m00s 1h00m00s - MX github.com. 42m35s 1 "aspmx.l.google.com." - MX github.com. 42m35s 10 "alt3.aspmx.l.google.com." - MX github.com. 42m35s 10 "alt4.aspmx.l.google.com." - MX github.com. 42m35s 5 "alt1.aspmx.l.google.com." - MX github.com. 42m35s 5 "alt2.aspmx.l.google.com." - NS github.com. 1h00m00s "dns1.p08.nsone.net." - NS github.com. 1h00m00s "dns2.p08.nsone.net." - NS github.com. 1h00m00s "dns3.p08.nsone.net." - NS github.com. 1h00m00s "dns4.p08.nsone.net." - NS github.com. 1h00m00s "ns-1283.awsdns-32.org." - NS github.com. 1h00m00s "ns-1707.awsdns-21.co.uk." - NS github.com. 1h00m00s "ns-421.awsdns-52.com." - NS github.com. 1h00m00s "ns-520.awsdns-01.net." -TXT github.com. 11m02s "MS=6BF03E6AF5CB689E315FB6199603BABF2C88D805" -TXT github.com. 11m02s "MS=ms44452932" -TXT github.com. 11m02s "MS=ms58704441" -TXT github.com. 11m02s "adobe-idp-site-verification=b92c9e999aef825edc36e0a3d847d2dbad5b2fc0e05c79ddd7a16139b48ecf4b" -TXT github.com. 11m02s "atlassian-domain-verification=jjgw98AKv2aeoYFxiL/VFaoyPkn3undEssTRuMg6C/3Fp/iqhkV4HVV7WjYlVeF8" -TXT github.com. 11m02s "docusign=087098e3-3d46-47b7-9b4e-8a23028154cd" -TXT github.com. 11m02s "stripe-verification=f88ef17321660a01bab1660454192e014defa29ba7b8de9633c69d6b4912217f" - TXT github.com. 11m02s "v=spf1 ip4:192.30.252.0/22 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com include:spf.protection.outlook.com include:mail.zendesk.com include:_spf.salesforce.com include:servers.mcsv.net ip4:166.78.69.169 ip4:166.78.69.170 ip4:166.78.71.131 ip4:167.89.101.2 ip4:167.89.101.192/28 ip4:192.254.112.60 ip4:192.254.112.98/31 ip4:192.254.113.10 ip4:192.254.113.101 ip4:192.254.114.176 ~all" -{{< /output >}} - -For reference, here are some of the most frequently seen DNS record types, along with brief introductions to each: - -- **A**: Contain the IPv4 addresses for hosts -- **AAAA**: Contain the IPv6 addresses for hosts -- **CNAME**: Keep aliases between domains -- **MX**: Name the mail server domains behind hosts -- **NS**: Give the nameservers responsible for hosts -- **TXT**: Hold arbitrary text for informational purposes - -You can see the list of record types supported by `dog` in its [official documentation](https://dns.lookup.dog/record-types). - -As with `dig`, `dog` gives you an option to output short records, using the `--short` flag. With this option, your results only include the main contents of the record — the IP address, for instance, in **A** records, the mail server in **MX** records, or the informational text in **TXT** records: - - dog github.com A --short - -{{< output >}} -192.0.2.0 -{{< /output >}} - -In addition to providing more readable output, `dog` also comes with an option to export your results as JSON. Here's an example that uses a query similar to the one above and saves the results directly as a `.json` file: - - dog github.com A NS TXT --json > dog-github-dns-lookup.json - -### Advanced Options - -Like `dig`, `dog` lets you specify a DNS server to use for your query. Domains typically have specifically delegated DNS servers that get used whenever you look up their records. However, you can use a tool like `dog` to conduct your lookup using an arbitrary DNS server, which can be useful for testing and troubleshooting: - - dog github.com @8.8.8.8 - -{{< output >}} -A github.com. 1m00s 192.0.2.0 -{{< /output >}} - -Both `dig` and `dog` support lookups for the TCP and UDP protocols. `dog` uses UDP by default, but you can easily use TCP by adding the `--tcp` flag to your command. - -However, in addition to these two protocols, `dog` adds options for two more: DNS over TLS (DoT) and DNS over HTTPS (DoH). Each of these protocols allows you to make more secure DNS queries. - -Here is an example that uses the DoT protocol via a Google DNS server. Using this option can mitigate threats of interference in the request and response: - - dog github.com MX --tls @dns.google - -{{< output >}} -MX github.com. 12m40s 1 "aspmx.l.google.com." -MX github.com. 12m40s 5 "alt1.aspmx.l.google.com." -MX github.com. 12m40s 5 "alt2.aspmx.l.google.com." -MX github.com. 12m40s 10 "alt3.aspmx.l.google.com." -MX github.com. 12m40s 10 "alt4.aspmx.l.google.com." -{{< /output >}} - -Below is an example using the DoH protocol via a Cloudflare DNS server. This protocol can be used for the same reason as the DoT protocol, but has the added feature that it runs on the popular **443** port. That potentially allows it to blend in with other traffic: - - dog github.com NS --https @https://cloudflare-dns.com/dns-query - -{{< output >}} -NS github.com. 6m21s "dns1.p08.nsone.net." -NS github.com. 6m21s "dns2.p08.nsone.net." -NS github.com. 6m21s "dns3.p08.nsone.net." -NS github.com. 6m21s "dns4.p08.nsone.net." -NS github.com. 6m21s "ns-1283.awsdns-32.org." -NS github.com. 6m21s "ns-1707.awsdns-21.co.uk." -NS github.com. 6m21s "ns-421.awsdns-52.com." -NS github.com. 6m21s "ns-520.awsdns-01.net." -{{< /output >}} - -## Conclusion - -To learn more about DNS, including more about record types and the role of DNS in the Internet, take a look at our guide [DNS Records: An Introduction](/cloud/guides/dns-overview). From there, you may also want to look at our guide [Troubleshooting DNS Records](https://techdocs.akamai.com/cloud-computing/docs/troubleshooting-dns-records). It can give you some ideas for how you might use a tool like `dog` to help keep your DNS setup in order. diff --git a/docs/guides/uptime/analytics/open-web-analytics-install-and-launch-on-your-server/index.md b/docs/guides/uptime/analytics/open-web-analytics-install-and-launch-on-your-server/index.md deleted file mode 100644 index 832bbe5b7ad..00000000000 --- a/docs/guides/uptime/analytics/open-web-analytics-install-and-launch-on-your-server/index.md +++ /dev/null @@ -1,194 +0,0 @@ ---- -slug: open-web-analytics-install-and-launch-on-your-server -title: 'Open Web Analytics (OWA): Install & Launch on Your Server' -description: This guide will show how to install Open Web Analytics (OWA) on CentOS 6.5, 7, Debian or Ubuntu. -authors: ["Douglas Colby"] -contributors: ["Douglas Colby"] -published: 2016-01-05 -keywords: ['open web analytics','owa','mysql','analytics'] -tags: ["centos","debian","ubuntu","analytics","statistics","mysql"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[OWA Website](http://www.openwebanalytics.com)' - - '[OWA Forum](http://www.openwebanalytics.com/?page_id=4)' -aliases: [] ---- - -Open Web Analytics (OWA) is an open-source alternative to commercial web analytics software. Use it to track and analyze traffic on your websites and applications. OWA analytics can easily be added to pages with simple Javascript, PHP, or REST based APIs. OWA also comes with built-in support for tracking websites made with popular content management frameworks such as WordPress and MediaWiki. - -## Before you Begin - -1. This guide assumes that you have your Linode already set up and running, that you have followed: - - - The [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide. - - The [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guides. - - That the Linode's [hostname is set](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and has a FQDN. - - - {{< note respectIndent=false >}} -The steps required in this guide require root privileges. Be sure to run the steps below as **root** or with the `sudo` prefix. For more information on privileges see our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. - -Your server must be configured with a fully qualified domain name (FQDN) and not just an IP address. If needed, you can use the address provided in the **Networking** tab next to your public IP address. -{{< /note >}} - - -2. Make sure your system is up-to-date: - - - **CentOS** - - yum update - - - **Debian & Ubuntu** - - apt-get update && apt-get upgrade - -3. Install the supporting software packages: - - - **CentOS** - - yum install httpd php php-mysql mysql-server mariadb-server - - {{< note respectIndent=false >}} -This command is designed to work with CentOS 6, which uses MySQL as the default database and CentOS 7 which uses MariaDB. You will get a notice when installing that the other package is not available. -{{< /note >}} - - - **Debian & Ubuntu** - - apt-get install apache2 php5 php5-mysql mysql-server - -4. CentOS users will need to enable and activate the `httpd` and `mariadb` services: - - - **CentOS 7** - - systemctl enable mariadb - systemctl start mariadb - systemctl enable httpd - systemctl start httpd - - - **CentOS 6** - - chkconfig mysqld on - service mysqld start - chkconfig httpd on - service httpd start - - Debian and Ubuntu users will need to restart the Apache2 daemon: - - service apache2 restart - -## Set up MySQL - -1. Run `mysql_secure_installation` to secure your database: - - mysql_secure_installation - - You should answer yes to most of the prompts. CentOS users, make sure to set a strong password for the root user since it is initially blank. This is the root user for `mysql` and is not related to the system's root user. - -2. Enter the MySQL CLI: - - mysql -u root -p - - You'll need to enter the password you set in the step above, or when you installed `mysql-server`. - -3. Create a database named `owadb`: - - {{< highlight sql >}} -CREATE DATABASE owadb; -{{< /highlight >}} - -4. Create a user named `owadbuser`. Replace the example password, `owadbpassword`, with a strong password of your choice. This information will be needed later to configure OWA. - - {{< highlight sql >}} -GRANT ALL PRIVILEGES ON owadb.* TO owadbuser@localhost IDENTIFIED BY 'owadbpassword'; -{{< /highlight >}} - -5. Exit the MySQL CLI: - - {{< highlight sql >}} -FLUSH PRIVILEGES; -{{< /highlight >}} - - quit - -## OWA - -### Install - -1. Navigate to your document root folder for your webserver, usually `/var/www/html` by default.: - - cd /var/www/html - -2. Download the OWA package: - - wget https://github.com/padams/Open-Web-Analytics/archive/1.5.7.tar.gz - - {{< note respectIndent=false >}} -Version 1.5.7 is the current version and may be different by the time you read this. Please check [The Open Web Analytics](http://www.openwebanalytics.com/) site for the latest information. -{{< /note >}} - -3. Unpack the downloaded file: - - tar xf 1.*.tar.gz - -4. Change ownership of the `owa` folder to the Apache daemon user: - - - **CentOS** - - chown -R apache:apache Open* - - - **Debian & Ubuntu** - - chown -R www-data:www-data Open* - -5. **Recommended:** Rename the OWA folder: - - mv Open-Web-Analytics-1.5.7 owa - -6. Delete the tar file: - - rm -rf 1.*.tar.gz - -### Configure - -1. Navigate to the OWA installation page in your web browser. Replace `your.domain` with your Linode's IP address or FQDN: - - http://your.domain/owa/ - - -2. After clicking on **Let's Get Started**, you should see a configuration page for your OWA installation. OWA will automatically fill in the first field with the path to your OWA installation. You will need to fill in the other fields on the page with the information you set in the MySQL CLI: - - ![Open Web Analytics set up screen.](owa-install.png) - -3. Click **Continue...**. - -4. Create a user account and define a domain to track. You will log in to see your OWA statistics through this user account. - - {{< note type="alert" respectIndent=false >}} -This process will display your password in plaintext once complete. Be careful if performing these steps in a public location. -{{< /note >}} - -## Using OWA - -You will need to create site profiles and add JavaScript or PHP code to your website pages to use OWA. - -1. Log in to your OWA installation. In your browser go to: - - http://your.domain/owa/index.php - -2. The first time you login OWA will present you with a tracking tag (a code snippet) for the domain you defined when you created your user account. This code should be added to the html pages of the site you wish to track. - -3. Once the tracking tag has been added to your website pages, analysis data can be viewed under the "Reporting" section (click the button at the top of the page) which will take you to the Sites Roster page. - -4. To add more sites to track, click "Add New" at the top left of the page. - -5. On the resulting page you will see near the top a section named "Add a New Tracked Site Profile". Enter the domain name of the site you want to track and click Save Profile. The other two fields are for your information only and are, therefore, optional. - - {{< note type="alert" respectIndent=false >}} -You must click "Save Profile" before trying to enter any of the settings below this button. Failure to do so will result in a blank white page and your new site not being added. Recovery simply requires a click of the back button in your browser. -{{< /note >}} - -6. Click on "Reporting" in the top left corner to return to the Sites Roster page. In the list of tracked sites, find your new site and click "Get Tracking Code". - -7. Copy the tracking code in the language of your choice (JavaScript or PHP) and paste the code into your websites' pages. Now whenever someone loads a page with the tracking code, OWA will know about it and the data will show up in the reports. - -8. Click "View Reports" in the Sites Roster page and begin happy analyzing! diff --git a/docs/guides/uptime/analytics/zipkin-server-configuration-using-docker-and-mysql/index.md b/docs/guides/uptime/analytics/zipkin-server-configuration-using-docker-and-mysql/index.md deleted file mode 100644 index 20d6af6c548..00000000000 --- a/docs/guides/uptime/analytics/zipkin-server-configuration-using-docker-and-mysql/index.md +++ /dev/null @@ -1,299 +0,0 @@ ---- -slug: zipkin-server-configuration-using-docker-and-mysql -title: 'Zipkin Server Configuration Using Docker and MySQL' -description: 'This guide shows you how to use Zipkin in a docker container for the purpose of tracking systems to collect and search timing data in order to identify latency problems.' -authors: ["Luis Cortés"] -contributors: ["Luis Cortés"] -published: 2017-10-04 -modified: 2017-10-27 -keywords: ["zipkin", "Docker", "tracking"] -tags: ["docker","statistics","fedora","analytics"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[Official ZipKin Documentation](http://zipkin.io/)' ---- - -[Zipkin](http://zipkin.io/) is a used for capturing timing data, it also has a centralized repository, and a microweb server that allows you to display, and search through spans and traces of your distributed programs or websites. - -We can configure Zipkin by [deploying it in a Docker container](http://zipkin.io/pages/quickstart). Using this approach, we can match the latest version of Zipkin by just pulling down the latest images. You can Isolate the Docker service and dependencies to just the container(s), and chose where you want your data to persist. Most importantly, by using Docker, you can spend more time focused on data analysis, instead of spending time on configuring a Zipkin as a service. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. This guide will use the hostname 'zipkinhost'. Also, do **not** follow the Configure a Firewall section yet. This guide includes firewall rules specifically for a Zipkin server. - -3. Assuming you know how to instrument a Zipkin client machine or have walked through our [Set Up a Zipkin Server with Sample Website Tracking](/cloud/guides/set-up-a-zipkin-server) guide. - -4. You will need an analyst system (laptop or workstation) with a web browser. This system will be used to view the traces/spans in the Zipkin server through the Zipkin provided webservice. - -While Zipkin can be installed on a variety of distributions, this guide uses Fedora 26 in the examples to configure both the server and client Linodes. Remember to adjust any distribution-specific commands, and replace the example IPs, `192.0.2.0` and `203.0.113.0`, with the IP addresses of your Zipkin server and analyst machine, respectively. - -## The Target Scenario - -Our main task is setting up a Zipkin server with MySQL, so that the spans/traces persist on the host file. - -## Zipkin Server Configuration - -### Install Package Dependencies - -1. Log into your Zipkin host machine and make sure your system is up to date: - - sudo dnf update && sudo dnf upgrade - -2. Add the Docker repository: - - sudo dnf install dnf-plugins-core - sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo - -2. Install Docker CE: - - sudo dnf install docker-ce - -3. Enable Docker as a service: - - sudo systemctl enable docker.service - sudo systemctl start docker.service - -4. (Optional) Add your limited user account to the `docker` group, so that you can run Docker commands without using `sudo`: - - sudo usermod -aG docker username - - You can test your Docker installation by running `docker run hello-world`. - -5. Install Docker Compose: - - curl -LO https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` - sudo mv docker-compose-Linux-x86_64 /usr/local/bin/docker-compose - sudo chmod +x /usr/local/bin/docker-compose - - You can test the installation with `docker-compose --version`. - - {{< note respectIndent=false >}} -The current stable version of Docker Compose is 1.16.1. Check for the latest version at the [releases page](https://github.com/docker/compose/releases) and update the version in the `curl` command accordingly. -{{< /note >}} - -6. Install git: - - sudo dnf install git - -7. Use git to retrieve the Zipkin Docker-compose YAML files at [openzipkin/docker-zipkin](https://github.com/openzipkin/docker-zipkin). This is one of the powerful features of Docker, these files hold all of the system level configurations we need, to run several different Zipkin configurations. Like, Zipkin with MySQL, Zipkin with elasticsearch, and Zipkin with Kakfa. - - cd ~ - git clone https://github.com/openzipkin/docker-zipkin.git - -8. Install MySQL: - - sudo dnf install mysql - -## Configure Docker - -The Docker service will manage your containers, the container's host, Zipkin services, and your MYSQL server. - -![host layout](zipkin_docker_host_layout.png) - -Docker is in charge of starting and stopping these services automatically when the host system is rebooted. it'll help us to map the ports from the container to the host's ports and it'll manage exporting the MySQL database files onto the host system. Docker can check to see if the container has failed, and restart it for us too. The host is in charge of running the actual Docker service and setting the firewall correctly. - -Notice that the Zipkin container will expose port 9411 for its service, and the MySQL container will expose port 3306. We'll use the Docker-compose YAML files to forward port 9411 to the host's port 9411, so that the container will be accessible on the internet. - -#### Zipkin Server Firewall Concepts - -To avoid the server being compromised, limit the exposure of our Zipkin server to just our analyst and client machines. - -The default Fedora 26 firewall rules block all ports as a safety precaution. Create a new firewall zone to handle the Zipkin services without exposing too much of the system: - -Our goal is to set up the Zipkin Server for: - -* Thrift service: receiving data from clients -* Web service: showing searches of time data -* lock down access to only our web and analyst machines. - -## Zipkin Server Firewall - -1. Create a new zone in our firewall called **zipkin** - - sudo firewall-cmd --new-zone=zipkin --permanent - -2. Reload the firewall and refresh your zone list. - - sudo firewall-cmd --reload - -3. Add an analyst machine IP (If you forget to define any source IPs, you will have no filtering on your IPs. You need at least 1 source IP to start filtering on IPs. If there are no source IPs defined, any machine can connect to your server.) - - sudo firewall-cmd --zone=zipkin --add-source=203.0.113.0/32 --permanent - -4. Open a port through your firewall. - - sudo firewall-cmd --zone=zipkin --add-port=9411/tcp --permanent - -5. (Optional) Since we may want to access our machine from the analyst machine, it may be a good idea to add an ssh port. - - sudo firewall-cmd --zone=zipkin --add-service=ssh --permanent - -6. Reload your firewall rules to activate them in your new zone. - - sudo firewall-cmd --reload - -7. View your new zone: - - sudo firewall-cmd --zone=zipkin --list-all - -## Docker-Compose Configuration - -The **docker-compose yml** files will control which system configuration we can use. We're going to select a MySQL configuration for storage. - -1. Copy the MySQL docker-compose YAML file to your home directory and rename it docker-init.yml as we're going to need to make a few changes: - - cd ~ - cp docker-zipkin/docker-compose.yml docker-init.yml - -2. Open `docker-init.yml` in a text editor and edit the content as follows: - - {{< file "~/docker-init.yml" >}} -version: '2' - -services: - storage: - image: openzipkin/zipkin-mysql - container_name: mysql - # Uncomment to expose the storage port for testing - ports: - - 3306:3306 - volumes: - - dbfiles:/mysql/data - restart: unless-stopped - - zipkin: - image: openzipkin/zipkin - container_name: zipkin - # Environment settings are defined here https://github.com/openzipkin/zipkin/tree/1.19.0/zipkin-server#environment-variables - environment: - - STORAGE_TYPE=mysql - # Point the zipkin at the storage backend - - MYSQL_HOST=mysql - # Uncomment to enable scribe - # - SCRIBE_ENABLED=true - # Uncomment to enable self-tracing - # - SELF_TRACING_ENABLED=true - # Uncomment to enable debug logging - # - JAVA_OPTS=-Dlogging.level.zipkin=DEBUG - ports: - # Port used for the Zipkin UI and HTTP Api - - 9411:9411 - depends_on: - - storage - restart: unless-stopped - - dependencies: - image: openzipkin/zipkin-dependencies - container_name: dependencies - entrypoint: crond -f - environment: - - STORAGE_TYPE=mysql - - MYSQL_HOST=mysql - # Add the baked-in username and password for the zipkin-mysql image - - MYSQL_USER=zipkin - - MYSQL_PASS=zipkin - # Uncomment to adjust memory used by the dependencies job - - JAVA_OPTS=-verbose:gc -Xms512m -Xmx512m - depends_on: - - storage - restart: unless-stopped - -volumes: - dbfiles: - -{{< /file >}} - - - - In the MySQL container section in the docker-init.yml, export the MySQL data directory, forward the MySQL port to the host, and add the restart command so that this service is automatically restarted if it goes down. - - - In the Zipkin container section in the docker-init.yml, make sure the port 9411 is forwarded to the host machine and add the restart command so that this service is automatically restarted if it goes down. - - - In the dependencies container, we uncomment the JAVA_OPTS and set it to at least 512M. This setting is optimized for a 1G Linode. However, if in the future this container needs more memory, you can increase this value. Add a restart command to the end of this section. - - -3. You can now update your Zipkin Docker images by performing a `docker pull` command. This will check the web for the images we need (the first time), and all other times it'll update the images to the latest version if need be. - - docker-compose -f docker-init.yml pull - -4. Run your Docker services by using the `docker-compose up` command. Conversely there is also a `docker-compose down` command that can be used to shutdown your Zipkin services. - - docker-compose -f docker-init.yml up -d - - Notice the **-d** flag at the end of the command, this **detaches** the container. Now it's running as its own process. If we just left the machine, it would continue to run, even if we rebooted the machine, it would run when the Docker service started it, because we didn't explicitly issue a `docker-compose down` command. Just for reference, to shutdown the Zipkin services: - - docker-compose -f docker-init.yml down - -## Backup Span/Trace Data - -There are 2 different backup methods: using MySQL , and using sysadmin. - -#### MySQL Backup - -1. Ensure that the MySQL service is running on a container. You can check this with a `docker ps` command. The `docker ps` command displays the active containers: - - {{< file "docker -ps" >}} -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -023d14e6193d openzipkin/zipkin-dependencies "crond -f" 3 days ago Up 3 days dependencies -ee0c255b7765 openzipkin/zipkin "/bin/sh -c 'test ..." 3 days ago Up 3 days 9410/tcp, 0.0.0.0:9411->9411/tcp zipkin -43f659b36f17 openzipkin/zipkin-mysql "/bin/sh -c /mysql..." 3 days ago Up 3 days 0.0.0.0:3306->3306/tcp mysql - -{{< /file >}} - -2. If isn't running, make sure you start the Zipkin services with the `docker-compose up` command first. Then issue the MySQLdump with the following parameters from your Zipkin host machine. - - mysqldump --protocol=tcp -A -pzipkin -uzipkin > ~/database.bak - - This command will dump the entire MySQL database from your MySQL container into the file called database.bak in your home directory. Alternatively, you can just dump your Zipkin span/trace data with: - - mysqldump --protocol=tcp -pzipkin -uzipkin zipkin > ~/db_zipkin.bak - -#### Database Backups - -We can just zip or tar the exported database files on the host system. Since we don't know if the container is writing information to these files at any given time, we need to make sure that the container is stopped. - -1. We can check the status with a `docker ps` command or just perform a `docker down` command. - - docker-compose -f docker-init.yml down - - After the `docker down` command, we can perform a `docker ps` command and see that there are no containers running. That should look like this: - - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - -2. At this point, we can create a zip backup of your files. The db files will be prepended with the name of your user. If you are running as root it would be **root_dbfiles**, but running as root isn't recommended. - - sudo zip -r ~/db_files.zip /var/lib/docker/volumes/_dbfiles/ - -3. Remember to start your Zipkin services if they're still needed. They will not restart even on a reboot because we have explicitly shut them down. - -## Testing the Zipkin Service - -1. Easiest way to do this is by using your web browser on your analyst machine. Log into your analyst machine, bring up your browser, and type in the following URL: - - http://192.0.2.0:9411/zipkin/ - - If you see the Zipkin web page, you're done. - -2. If you don't see a web page, log into the Zipkin host machine, and make sure your containers are up by running either `docker ps` command or `docker-compose up` command. If the containers are not all running, it's possible that your Linode has run out of memory. - -3. Make sure your firewall port is open by typing: - - sudo firewall-cmd --add-port 9411/tcp --permanent - -4. At this point, what might have happened is that we added the wrong IP address of our analyst machine. We can check this by logging into our analyst machine. If our analyst machine is a Fedora workstation, we can install **nmap** and perform a network port status check to our Zipkin host machine like so: - - sudo dnf install -y nmap - nmap 192.0.2.0 -p 9411 -Pn - - A good return has an **open** for the STATE of the port, anything else and we probably don't have the right analyst machine IP address in our firewall rules: - - Starting Nmap 7.40 (https://nmap.org) at 2017-09-24 18:34 MDT - Nmap scan report for zipkin (192.0.2.0) - Host is up (0.10s latency). - PORT STATE SERVICE - 9411/tcp open unknown - - Review your firewall rules and try again. - diff --git a/docs/guides/uptime/monitoring/adagios-web-interface-for-nagios/index.md b/docs/guides/uptime/monitoring/adagios-web-interface-for-nagios/index.md deleted file mode 100644 index 60d8ea9a734..00000000000 --- a/docs/guides/uptime/monitoring/adagios-web-interface-for-nagios/index.md +++ /dev/null @@ -1,257 +0,0 @@ ---- -slug: adagios-web-interface-for-nagios -title: "Setting Up the Adagios Web Interface for Nagios" -description: "Adagios provides a clean and simplified approach to using the Nagios server-monitoring system. Learn everything you need to get started with Adagios in this tutorial." -authors: ['Nathaniel Stickman'] -contributors: ['Nathaniel Stickman'] -published: 2023-06-21 -keywords: ['adagios tutorial','nagios monitoring','nagios web interface'] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[GitHub: opinkerfi/adagios - Users Guide](https://github.com/opinkerfi/adagios/wiki/Users-guide)' -- '[GitHub: opinkerfi/adagios - Using Adagios WEB API](https://github.com/opinkerfi/adagios/wiki/Using-Adagios-WEB-API)' ---- - -Adagios offers a more intuitive interface for observing and configuring the Nagios server-monitoring system. Adagios has the key features from Nagios readily available, while keeping the messier details out of the way. - -In this tutorial, learn how to get started with Adagios on a CentOS Stream 8 or similar system. Follow along to install Adagios and start using it to monitor your servers. - -## Before You Begin - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - -1. Your Linode Compute Instance needs to be running CentOS 8 or a comparable distribution. The instructions in this tutorial have been specifically tested on CentOS 8, but should be compatible with systems like AlmaLinux 8 and Rocky Linux 8 as well. - -1. Your Compute Instance needs to be on the Linode 8 GB Shared CPU plan, at minimum, as these instructions require a large amount of disk space. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, see the [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Getting Started with Nagios - -Before diving into Adagios, you should have a base familiarity with Nagios, the tool that Adagios provides an interface for. If you are not familiar with Nagios, continue reading and follow the links below to learn more. - -### What Is Nagios? - -[Nagios](https://www.nagios.com/products/nagios-core/) is a widely used server-monitoring tool. Nagios provides robust server monitoring with highly configurable notification and alert options along with a host of official and community plugins. - -### How to Install Nagios - -Before getting started with Adagios, Nagios must be installed and configured on your system. To install Nagios, follow our [How to Install Nagios on CentOS 8](/cloud/guides/install-nagios-on-centos-8) guide. - -Alternatively, refer to the official [installation documentation](https://support.nagios.com/kb/article/nagios-core-installing-nagios-core-from-source-96.html) for Nagios Core for details on installing Nagios on other operating systems. - -Keep in mind that Adagios is built primarily with CentOS and other RHEL-based systems in mind. For that reason, this tutorial focuses on CentOS Stream 8. The instructions should work similarly for distributions like AlmaLinux 8 and Rocky Linux 8. - -## How to Start Using Adagios with Nagios - -Adagios runs on top of Nagios to provide a cleaner and simpler interface for managing a Nagios server-monitoring instance. Follow along with this section of the tutorial to learn more about Adagios and get a full walkthrough for setting it up. - -### Why Use Adagios? - -[Adagios](http://adagios.org/) is a web interface for managing Nagios configurations. While Nagios comes with its own web interface, Adagios offers a more approachable dashboard for monitoring performance and managing a Nagios configuration. Adagios hides more of the low-level details, making it easier to get a quick view of a system and control the monitoring configuration. - -Additionally, Adagios provides a RESTful API for accessing and managing a Nagios instance. This makes Adagios a broadly useful enhancement to the default interfaces offered by Nagios. - -### Installing Adagios - -On most systems, the best way to install Adagios is from source. This approach makes the Adagios configuration easier to manipulate, which is necessary depending on where Nagios is installed. - -#### Setting Up the Prerequisites - -1. First, install dependencies for Adagios from the package manager: - - ```command - sudo dnf install epel-release - ``` - - ```command - sudo dnf config-manager --set-enabled powertools - sudo dnf install git acl mod_ssl python36 python36-devel python3-pip python3-setuptools make gcc-c++ - ``` - - ```command - sudo pip3 install --upgrade pip - ``` - -1. Install the Livestatus add-on for Nagios. The add-on provides features to enhance how status information about hosts and services are retrieved. Opin Kerfi, the maintainers of Adagios, also maintain a package to conveniently pull the Livestatus installation files: - - ```command - cd /tmp - wget https://droppa.ok.is/dl/data/8b15d652f132667abbf0b80dd9d956d4cc368d1c/mk-livestatus-1.2.6.tar.gz - tar -xvzf mk-livestatus-1.2.6.tar.gz - cd mk-livestatus-1.2.6 - sudo ./configure --with-nagios4 - sudo make - sudo make install - ``` - -1. Complete the Livestatus installation by creating a directory for Nagios to store the Livestatus socket and placing that information in the Nagios configuration file: - - ```command - sudo mkdir -p /usr/lib/nagios/mk-livestatus - sudo chown nagios:apache /usr/lib/nagios/mk-livestatus - echo 'broker_module=/usr/local/lib/mk-livestatus/livestatus.o /usr/lib/nagios/mk-livestatus/livestatus' | sudo tee -a /usr/local/nagios/etc/nagios.cfg - ``` - -1. Create a user group and user for Adagios, then assign the Nagios user group to that user. - - ```command - sudo groupadd --system adagios - sudo adduser --system -g adagios adagios - sudo usermod -aG nagios adagios - ``` - -#### Installing and Configuring Adagios - -1. Download the Adagios repository and give its ownership to the Adagios user: - - ```command - sudo git clone -b master --depth 1 https://github.com/opinkerfi/adagios.git /opt/adagios - sudo chown --recursive adagios:adagios /opt/adagios/ - ``` - -1. Prepare directories for Adagios, and move its configuration files onto your system: - - ```command - sudo mkdir -p /etc/adagios/conf.d /var/lib/adagios /usr/local/nagios/etc/adagios - sudo cp /opt/adagios/adagios/etc/adagios/adagios.conf /etc/adagios/adagios.conf - ``` - -1. Adjust the Adagios configuration files to reflect the actual location of your Nagios installation. - - The changes here assume your `nagios.cfg` file is stored within the `/usr/local/nagios/etc/` directory. That should be the case when following the Nagios installation guide linked earlier in this tutorial. However, replace `/usr/local/nagios/` below with the actual directory containing your installed Nagios files if that directory differs. - - These changes also disable PNP4Nagios, which adds performance graphing features. However PNP4Nagios is now deprecated, so it is not covered here. To install it anyway, follow [Nagios's instructions](https://support.nagios.com/kb/article/nagios-core-performance-graphs-using-pnp4nagios-801.html#CentOS) and do not execute the `enable_pnp4nagios` command below. - - ```command - sudo sed -i 's|/etc/nagios|/usr/local/nagios/etc|g' /etc/adagios/adagios.conf - sudo sed -i 's|/usr/sbin/nagios|/usr/local/nagios/bin/nagios|g' /etc/adagios/adagios.conf - sudo sed -i 's|enable_pnp4nagios = True|enable_pnp4nagios = False|g' /etc/adagios/adagios.conf - sudo sed -i 's|# ALLOWED_HOSTS|ALLOWED_HOSTS|g' /etc/adagios/adagios.conf - sudo sed -i 's|livestatus_path = None|livestatus_path = "/usr/lib/nagios/mk-livestatus/livestatus"|g' /etc/adagios/adagios.conf - ``` - -1. Copy the remaining Adagios configuration files to the appropriate places on your system, and ensure the Adagios user has the ownership it needs: - - ```command - sudo cp /opt/adagios/adagios/etc/adagios/conf.d/force_script_name.conf /etc/adagios/conf.d/ - sudo cp /opt/adagios/adagios/etc/adagios/conf.d/okconfig.conf /etc/adagios/conf.d/ - sudo cp /opt/adagios/adagios/etc/sudoers.d/adagios /etc/sudoers.d/ - sudo chown --recursive adagios:adagios /etc/adagios /var/lib/adagios - ``` - -1. Set up the Adagios service files, then start and enable the Adagios service: - - ```command - sudo cp /opt/adagios/contrib/gunicorn.py /opt/adagios/gunicorn.py - sudo cp /opt/adagios/contrib/*.service /etc/systemd/system/ - sudo systemctl daemon-reload - sudo systemctl start adagios - sudo systemctl enable adagios - ``` - -1. Adagios includes a configuration file for your Apache Web Server instance. Copy that configuration file to the directory for Apache configurations: - - ```command - sudo cp /opt/adagios/contrib/apache_gunicorn_adagios.conf /etc/httpd/conf.d/ - ``` - -1. Adagios uses a Python virtual environment for orchestrating some of its required features. Execute the included script to set up the necessary virtual environment, then load the environment and run some commands to provide initial configurations. - - Again, you need to replace `/usr/local/nagios/` below with the actual location of your installed Nagios files if they differ. - - ```command - sudo /opt/adagios/upgrade.sh - sudo chown --recursive adagios:adagios /opt/adagios/ - su - - ``` - - ```command - source /opt/adagios/venv/bin/activate - pynag config --append cfg_dir=/usr/local/nagios/etc/adagios - pynag config --append "broker_module=/usr/local/lib/mk-livestatus/livestatus.o /usr/lib/nagios/mk-livestatus/livestatus'" - exit - ``` - -1. Restart the Nagios, Adagios, and Apache services for the configuration changes to take effect: - - ```command - sudo systemctl restart nagios adagios httpd - ``` - -#### Installing and Configuring OKConfig - -Adagios leverages the [OKConfig](https://github.com/opinkerfi/okconfig) tool for managing Nagios configurations. Although not necessary, you should install and configure OKConfig to get the most out of Adagios. - -1. Download the OKConfig repository and deploy its initial configurations: - - ```command - sudo git clone -b master --depth 1 https://github.com/opinkerfi/okconfig.git /opt/okconfig - echo 'export PYTHONPATH=$PYTHONPATH:/opt/okconfig' | sudo tee /etc/profile.d/okconfig.sh - sudo cp /opt/okconfig/etc/okconfig.conf /etc/okconfig.conf - source /etc/profile - sudo ln -s /opt/okconfig/usr/share/okconfig /usr/share/ - sudo ln -s /opt/okconfig/usr/bin/okconfig /usr/local/bin/ - ``` - -1. Make some changes to the default OKConfig configuration. As with the Adagios configuration changes above, these primarily concern pointing OKConfig to the actual location of the `nagios.cfg` file. - - Adjust the directories in these commands to match the location of your Nagios installation if its files are stored elsewhere. - - ```command - sudo sed -i 's|/etc/nagios/nagios.cfg|/usr/local/nagios/etc/nagios.cfg|g' /etc/okconfig.conf - sudo sed -i 's|/etc/nagios/okconfig|/usr/local/nagios/etc/okconfig|g' /etc/okconfig.conf - echo 'cfg_dir=/usr/share/okconfig/templates' | sudo tee -a /usr/local/nagios/etc/nagios.cfg - su - - ``` - -1. Start up OKConfig by entering the Adagios virtual environment and running the OKConfig commands: - - ```command - source /opt/adagios/venv/bin/activate - okconfig init - okconfig verify - exit - ``` - -1. Restart the Nagios, Adagios, and Apache services for the changes to take effect: - - ```command - sudo systemctl restart nagios adagios httpd - ``` - -### Navigating Adagios - -With Adagios installed and running, you now have access to the Adagios interface for your Nagios instance. Open a web browser and navigate to the `/adagios/` path on your system's public address. - -For instance, if your public IP address is `192.0.2.0`, navigate to `http://192.0.2.0/adagios/`. If your system has the domain name `example.com`, navigate to `http://example.com/adagios/`. - -You should be greeted with the Adagios dashboard: - -[![The Adagios dashboard](adagios-dashboard_small.png)](adagios-dashboard.png) - -From here, use Adagios to navigate the myriad information about your monitored hosts and services. More details are available through the options on the left menu, such as navigating to the **Services** page to see a list of monitored services: - -[![Services listed in Adagios](adagios-services_small.png)](adagios-services.png) - -Individual items can be selected for even more details. - -Use the **Configuration** option from the top of the page to see options for configuring hosts, services, commands, and more within Nagios: - -[![Adagios page for configuring services](adagios-configure_small.png)](adagios-configure.png) - -The top menu also has an option for **Nagios**, providing an inset window with the traditional Nagios interface should you need specific features it provides. - -Finally, navigating to the `/adagios/rest` path opens the menu for documentation of your Adagios instance's RESTful APIs. There are several options of APIs exposed by the Adagios instance. For example, here is how the **status** API may look: - -[![Adagios page documenting the Status REST API](adagios-rest-api_small.png)](adagios-rest-api.png) - -## Conclusion - -Now you have Adagios providing a clean and easy interface for your Nagios system. Be sure to look into all of the menu options the Adagios interface offers. This should give you a thorough idea of the features Adagios offers and how to make the most of them. - -To take your Adagios setup even further, follow along with the Adagios user documentation linked below. The documentation on using the REST API features is especially helpful should you want to leverage them. \ No newline at end of file diff --git a/docs/guides/uptime/monitoring/logwatch-monitor-system-logs/index.md b/docs/guides/uptime/monitoring/logwatch-monitor-system-logs/index.md deleted file mode 100644 index de74eaf3671..00000000000 --- a/docs/guides/uptime/monitoring/logwatch-monitor-system-logs/index.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -slug: logwatch-monitor-system-logs -title: "Monitor System Logs with Logwatch" -description: "This guide shows you how to use logwatch, a Linux utility used to monitor system logs and create reports for critical and non-critical events." -authors: ["Elle Krout"] -contributors: ["Elle Krout"] -published: 2015-08-12 -keywords: ["logwatch", "security", "logging", "audit", "logs"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -tags: ["monitoring"] -aliases: [] -image: monitor-system-logs-logwatch.jpg ---- - -*Logwatch* is a log parsing program that analyzes and generates daily reports on your system's log activity. Logwatch does not provide real time alerts, but instead creates a digest organized by systems for ease of review. More advanced users can also pair Logwatch with custom analysis scripts to fine-tune their reports. - -By default, Logwatch uses Sendmail to send digests. - -{{< note >}} -The steps required in this guide require root privileges. Be sure to run the steps below as **root**. If logged in as a superuser, it is recommended that you `su` into root. For more information on privileges see our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Install Logwatch and Sendmail - -### Ubuntu - -1. Update your system: - - sudo apt update && sudo apt upgrade - -1. Install Logwatch and Sendmail: - - sudo apt install logwatch sendmail - -### Debian - -1. Update your system: - - sudo apt update && sudo apt upgrade - -2. Install Logwatch and Sendmail: - - sudo apt install logwatch sendmail-bin sendmail - -### CentOS Stream, AlmaLinux, Rocky Linux, and Fedora - -1. Update your system: - - sudo dnf update - -2. Install Logwatch and Sendmail: - - sudo dnf install logwatch sendmail - -3. Start Sendmail: - - sudo systemctl start sendmail - -### CentOS 7 - -1. Update your system: - - sudo yum update - -2. Install Logwatch and Sendmail: - - sudo yum install logwatch sendmail - -3. Start Sendmail: - - sudo systemctl start sendmail - -### Arch Linux - -1. Update your system: - - pacman -Syu - -2. Install Logwatch and Postfix, to replace the default Sendmail, which is not in Arch's repositories. - - pacman -S logwatch postfix - - Logwatch will prompt you to select which cron provider to use. Select the default, *cronie*. - - {{< note respectIndent=false >}} -Other SMTP clients can also be used for delivering Logwatch messages. -{{< /note >}} - -3. Edit the `/etc/postfix/main.cf` file to add your domain information, and allow for send-only mail, replacing `hostname.example.com` with your own hostname and domain: - - {{< file "/etc/postfix/main.cf" aconf >}} -myhostname = hostname.example.com -inet_interfaces = loopback-only -{{< /file >}} - - {{< note respectIndent=false >}} -Both A/AAAA, and MX records will need to be set for your domain. -{{< /note >}} - -4. Edit `/etc/postfix/aliases` to uncomment `root` and alias it to `root@hostname.example.com`, replacing `hostname.example.com` with your own hostname and domain: - - {{< file "/etc/postfix/aliases" >}} -root: root@hostname.example.com -{{< /file >}} - -5. Run `newaliases` after editing the aliases list. - -6. Start postfix: - - systemctl start postfix - -## Configure Logwatch - -The default configuration file for Logwatch is located at `/usr/share/logwatch/default.conf/logwatch.conf`. This file contains information on which directories for Logwatch to track, how the digest is output, where the digest is sent to, and which services of which to keep track. - -The following settings are the most comment configuration changes that will need to be made. Others can be found in the `logwatch.conf` file, explained in the comments. - -{{< note >}} -If Logwatch initially does not appear to run, within the `logwatch.conf` file, change the `Details` setting to `Med`. -{{< /note >}} - -### Log Directories - -By default, Logwatch digests will include all logs contained within `/var/log`. If any other directories contain logs, such as website directories, they can be added by including additional `LogDir` lines. For example: - -{{< file "/usr/share/logwatch/default.conf/logwatch.conf" >}} -LogDir = /var/log -LogDir = /var/www/example.com/logs - - -{{< /file >}} - - -### Print Logwatch Digest to Console - -The default Logwatch configuration will output the digest to your Linode's console. This is defined with the `Output` variable, which is set to `stdout` by default. This option is feasible if you are only planning on manually running Logwatch, but does not save or send the logs to you for later perusal. - - -### Email Logwatch Digest - -The Logwatch digest can be sent to local users or external email addresses, in plain text or HTML formats. - -{{< note >}} -Prior to sending mail externally or locally ensure you have Sendmail installed on the Linode. If you choose to use a different MTA client, change the `mailer` line in the Logwatch configuration file to contain the directory of your chosen MTA, or alias `/usr/sbin/sendmail` to your MTA. - -If using Arch, and you followed the above install instructions, Sendmail is already aliased to msmtp. -{{< /note >}} - -1. Change the `Output` value to `mail`. If you wish to receive the messages in HTML format change the `Format` value to `html`. - -2. Change the `MailTo` address to a valid email address, or local account user. For example, to send mail to the `root` user change the line to read: - - {{< file "/usr/share/logwatch/default.conf/logwatch.conf" >}} -MailTo = root - - -{{< /file >}} - - -3. Change the `MailFrom` value to a valid email address, or to a local user. This can also be left as `Logwatch`. - - -### Save Logwatch Digest to File - -Logwatch digests can also be saved to a file on your system. - -1. Change the `Output` value to `file`. - -2. Find and uncomment (remove the hashmark [**#**]) the `Filename` value. Set the path and filename in which you wish to save your Logwatch digests. - - -## Run Logwatch - -### Run Logwatch Manually - -Logwatch can be run manually at any time by inputting the `logwatch` command to your console. This command can be appended with a number of options to change the default output to suit your needs: - -- `--detail`: Can be set to low, med, high, or any numerical values between 1 and 10. Defines how detailed the report will be. -- `--logdir`: The directory containing the log files you wish to gain reports on. -- `--service`: The service definition that you wish to report on. -- `--output`: How you want the file to be sent: Standard output (`stdout`), mail, or file. -- `--format`: Plain text or HTML. -- `--mailto`: The local user or email address to send the report to. - -### Run Logwatch through Cron - -Logwatch often works best when configured to run daily and send or save a report to view later. This can be achieved by setting Logwatch up to run as a cronjob. - -1. Open the crontab: - - crontab -e - -2. Add a line for Logwatch. The following code is configured to run at 00:30 each day: - - {{< file "/etc/crontab" >}} -30 0 * * * /usr/sbin/logwatch - - -{{< /file >}} - - - For more information on adjusting your crontab scheduling, reference our guide on [Scheduling Tasks with Cron](/cloud/guides/schedule-tasks-with-cron). diff --git a/docs/guides/uptime/monitoring/monitor-remote-hosts-with-icinga/index.md b/docs/guides/uptime/monitoring/monitor-remote-hosts-with-icinga/index.md deleted file mode 100644 index cf755fdef9d..00000000000 --- a/docs/guides/uptime/monitoring/monitor-remote-hosts-with-icinga/index.md +++ /dev/null @@ -1,215 +0,0 @@ ---- -slug: monitor-remote-hosts-with-icinga -title: 'Monitor Remote Hosts with Icinga' -description: "This guide shows how to configure Icinga 2 to monitor remote systems on your Linode" -og_description: "This guide will show you how to configure Icinga 2 to monitor your remote systems. Icinga 2 can monitor local and remote systems, and this guide shows you how to do both." -authors: ["Matt Vass"] -contributors: ["Matt Vass"] -published: 2017-12-28 -modified: 2017-12-19 -keywords: ["debian", "icinga", "monitoring", "icinga2"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Official Icinga Documentation](https://www.icinga.com/docs/icinga2/latest/doc/01-about/)' -tags: ["monitoring"] -aliases: [] ---- - -![Monitor Remote Hosts with Icinga](monitor-remote-hosts-icinga-title.jpg "Monitor Remote Hosts with Icinga") - -## What is Icinga 2? - -This guide is a continuation of our guide on [Icinga 2](/cloud/guides/install-icinga2-monitoring-on-debian-9). -Icinga, is an open source network monitoring application that can be used to monitor critical services and systems on your Linode. Icinga 2 can monitor hosts on a network or it can verify network external protocols, such as the state of an HTTP server, mail server, file-sharing service, or others. - -## Before You Begin - -The steps and examples in this guide assume the defaults and configurations from the previous guide. Adjust all variables accordingly. - -## Monitor Remote Hosts via Simple Host Monitoring - -In order to monitor a host and its external services via regular command checks, Icinga 2 uses a mechanism that issues a ping command against the server's IP address at regular intervals and using its built-in commands, regularly verifies the state of remote network services protocols, such as HTTP, SSH, SMTP, IMAP, POP or others.\ -Icinga 2 stores Host definitions in objects. These objects and their attributes are used for applying rules for `Service`, `Notification`, `Dependency`, and `Scheduled Downtime` can be found in `hosts.conf` file, in `/etc/icinga2/conf.d/`. - -1. To add a new host definition to be periodically monitored by Icinga 2 engine via ICMP checks, open `hosts.conf` and add the following lines to the bottom of the file: - - {{< file "/etc/icinga2/conf.d/hosts.conf" conf >}} -object Host "Linode" { - import "generic-host" - address = "10.25.1.31" - check_command = "hostalive" -} -{{< /file >}} - -2. To check the status of a web server that runs on this node, add the following lines after host definition. This check will verify if the web server is alive and responds with the proper HTTP codes: - - {{< file "/etc/icinga2/conf.d/hosts.conf" conf >}} -object Service "http" { - host_name = "Linode" - check_command = "http" -} -{{< /file >}} - -3. Restart the service to apply the host definitions and start monitoring the new host resource: - - systemctl restart icinga2.service - -4. Verify the state of the new host by navigating to **Overview** then **Hosts** in Icinga’s Web 2 interface. - -5. To display the status of the host’s HTTP service, navigate to **Overview** then **Servicegroups** and click **HTTP Checks**. - -## Monitor Remote Hosts via Icinga 2 Agent Monitoring - -Icina2 can monitor a node's internal health parameters, such as CPU load, disk space, memory, and number of running process via a secured channel set up between a master node and client node on port `5665/TCP`. In this instance we’ll configure our Icinga 2 to act as the master node and monitor the remote CentOS 7 client node. In this specific type of configuration, also called a *Top Down Command Endpoint* model, the check commands will be scheduled on the master node and then will be sent to the client via a TLS connection. - -1. Set up the Icinga 2 master node on our Debian 9 server. Configure this instance of Icinga 2 as a master node: - - icinga2 node wizard - -2. Follow the prompts, and enter `n` to install a master setup: - - Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]: n - -3. Press **Enter** to accept the default for each of the following: - - Please specify the common name (CN) [icinga]: - Please specify the API bind host/port (optional): - Bind Host []: - Bind Port []: - -4. Restart the Icinga 2 service to apply the master node configuration and check the daemon status: - - systemctl restart icinga2.service - systemctl status icinga2.service - -5. Output the port number of this master node and open the port in your Debian firewall: - - netstat -tulpn| grep icinga - -6. Generate a client ticket for your client node. Use the hostname of your client to generate the ticket. In this example the client node hostname is `centos`. Replace the hostname of the client and make sure you run the command with `root` privileges: - - icinga2 pki ticket --cn 'centos' - - The command will generate and display a key. Copy or note this key because you will need it to set up the CentOS client later. - -## Configure CentOS 7 Client Node - -1. Log in to your CentOS 7 system with an account with `root` privileges or directly as root and issue the following command to enable EPEL and Icinga 2 repositories in CentOS. Also, make sure your CentOS 7 system is configured with a static IP address. - - yum install epel-release - yum install https://packages.icinga.com/epel/icinga-rpm-release-7-latest.noarch.rpm - -2. Install the Igina2 engine and Nagios plugins required by Icinga 2 to execute the check commands in CentOS by issuing the following command: - - yum install icinga2 nagios-plugins-all - -3. After the Icinga 2 daemon has been installed in your CentOS system, start the node wizard and configure this system as a satellite node instead of master node: - - icinga2 node wizard - -4. Use the client node wizard to configure the CentOS client node with the following answers when prompted. Press **Enter** to accept the default as shown below: - - Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]: y - Please specify the common name (CN) [centos]: **Enter** - Master Common Name (CN from your master setup): icinga - Do you want to establish a connection to the master from this node? [Y/n]: y - Master endpoint host (Your master's IP address or FQDN): 10.25.32.120 - Master endpoint port [5665]: **Enter** - Add more master endpoints? [y/N]: n - Please specify the master connection for CSR auto-signing (defaults to master endpoint host): **Enter** - Host [10.25.32.120]: **Enter** - Port [5665]: **Enter** - -5. Verify the information when prompted: - - Is this information correct? [y/N]: y - -6. Enter the ticket key generated in Step 6 of the previous section: - - Please specify the request ticket generated on your Icinga 2 master. - (Hint: # icinga2 pki ticket --cn 'centos'): a34c7fcbc4f5311257d2a5d4e7f1961dc3c64cb5 (Enter the ticket key generated on the master node) - -7. Complete the installation: - - Please specify the API bind host/port (optional): **Enter** - Bind Host []: **Enter** - Bind Port []: **Enter** - Accept config from master? [y/N]: y - Accept commands from master? [y/N]: y - -8. After the client node wizard completes, restart the Icinga 2 service, check Icinga 2 service status, list Icinga’s listening port, and add the Icinga 2 listening port number to the CentOS firewall: - - systemctl restart icinga2 - systemctl status icinga2 - ss –tlpn|grep icinga2 - firewall-cmd --add-port=5665/tcp --permanent - firewall-cmd --reload - -## Set up Icinga 2 Master Agent-based Monitoring - -1. Log in to the Icinga 2 master node and create a CentOS client zone directory, a client configuration, and a services file: - - mkdir /etc/icinga2/zones.d/centos/ - touch /etc/icinga2/zones.d/centos/centos.conf - touch /etc/icinga2/zones.d/centos/services.conf - -2. Add the following lines to the `centos.conf` zone configuration file: - - {{< file "centos.conf" conf >}} -object Zone "centos" { - endpoints = [ "centos" ] - parent = "icinga" -} - -object Endpoint "centos" { - host = "192.168.1.100" -} - -object Host "centos" { - import "generic-host" - address = "192.168.1.100" - vars.os = "Linux" - vars.notification["mail"] = { - groups = [ "icingaadmins" ] - } - vars.client_endpoint = name -} -{{< /file >}} - - The CentOS endpoint object zone will report back to its master node defined by the `parent = “icinga”` statement. Replace the CentOS hostname and IP address accordingly. - -3. Update the CentOS zone services configuration file: - - {{< file "/etc/icinga2/zones.d/centos/services.conf" conf >}} -apply Service "users" { - import "generic-service" - check_command = "users" - command_endpoint = host.vars.client_endpoint - assign where host.vars.client_endpoint -} - -apply Service "procs" { - import "generic-service" - check_command = "procs" - command_endpoint = host.vars.client_endpoint - assign where host.vars.client_endpoint -} -{{< /file >}} - - In this configuration file we’ve defined the following services checks for the remote client: - - - * Verify number of users logged in to the system and the number of processes running. - * The `command_endpoint` lines force the service checks to be transmitted to the remote CentOS system and executed by the Icinga 2 engine command endpoint. - * You can add as many commands as you’d like here to be executed internally on the remote host. However, if Icinga sent instructions are not present on the remote node as Nagios plugin scripts, the commands won’t execute and an error will be displayed in the icinga2 web interface. - - -4. Restart the icinga2 service to apply the configurations. - -5. Navigate to the Icinga Web 2 interface to verify that the command checks are running on the remote node host. - -## That’s all! - -You have successfully configured Icinga 2 as a master node and added a CentOS 7 client node to be remotely checked via Icinga 2 agent-based monitoring system and another remote host to be actively monitored via external services command checks. - -For other Icinga 2 configurations, installation, and monitoring mechanisms, visit the [official Icinga 2 documentation](https://www.icinga.com/docs/icinga2/latest/doc/01-about/). diff --git a/docs/guides/uptime/monitoring/ossec-ids-debian-7/index.md b/docs/guides/uptime/monitoring/ossec-ids-debian-7/index.md deleted file mode 100644 index 09ac2b50936..00000000000 --- a/docs/guides/uptime/monitoring/ossec-ids-debian-7/index.md +++ /dev/null @@ -1,492 +0,0 @@ ---- -slug: ossec-ids-debian-7 -title: Install and Configure OSSEC on Debian 7 -description: 'This guide shows how to install and configure OSSEC HIDS, an open-source, host-based intrusion detection system which provides advanced monitoring functions, on Debian 7.' -authors: ["Sunday Ogwu-Chinuwa"] -contributors: ["Sunday Ogwu-Chinuwa"] -published: 2015-02-05 -keywords: ["IDS", " Intrusion Detection System"] -aliases: [] -tags: ["debian","monitoring"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[OSSEC](https://ossec.github.io/)' ---- - -![Install and Configure OSSEC on Debian 7](Install-and-Configure-OSSEC-on-Debian-7-smg.jpg) - -OSSEC is an open-source, host-based intrusion detection system (**HIDS**) that performs log analysis, integrity checking, rootkit detection, time-based alerting, and active response, making it an ideal choice for server monitoring. - -When installed and configured, OSSEC will provide a real-time view of what's taking place in your server or servers in a server/agent mode. - -This guide covers how to install and configure OSSEC on a single Linode running Debian 7 in such a manner that if a file is modified, added or deleted, OSSEC will notify you by email in real-time. OSSEC can also provide notifications for other activities. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Prerequisites - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. It is especially important to create and enable a firewall. On Linux, OSSEC needs an active iptables firewall for its active response feature to work. - -3. Install `inotify-tools` and `build-essential`. OSSEC needs the first for real-time alerts and alerts on file deletions, while the latter is for compiling OSSEC. You can install both packages using: - - sudo apt-get install inotify-tools build-essential - -## Download and Verify OSSEC - -Debian 7 does not have an installation candidate for OSSEC in its repository, so it will need to be downloaded from the project's [download page](http://ossec.github.io/downloads.html). - -1. Download the latest OSSEC tarball (2.8.3 at the time of this guide's publication): - - wget -U https://bintray.com/artifact/download/ossec/ossec-hids/ossec-hids-2.8.3.tar.gz - -2. Download the checksum file: - - wget -U http://ossec.wazuh.com/vm/ossec-vm-2.8.3-checksum.txt - -3. Verify that the downloaded tarball has not been tampered with using the MD5 checksum: - - md5sum -c ossec-vm-2.8.3-checksum.txt - - The expected output is: - - ossec-hids-2.8.3.tar.gz: OK - md5sum: WARNING: 1 line is improperly formatted - -4. Verify the SHA1 checksum: - - sha1sum -c ossec-hids-2.8.3-checksum.txt - - The expected output is: - - ossec-hids-2.8.3.tar.gz: OK - sha1sum: WARNING: 1 line is improperly formatted - - - {{< note respectIndent=false >}} -In both outputs, ignore the **WARNING** line. As long as the first line reads **OK**, the file is good. -{{< /note >}} - -## Install OSSEC - -1. Untar the file: - - tar xf ossec-hids-2.8.3.tar.gz - -2. Change to the newly-created directory, then run the install: - - cd ossec-hids-2.8.3 - sudo ./install.sh - -3. Select your language. If your language is the default (English - [en]), press `Return`; otherwise, type in the 2-letter code for your language from the list of supported languages: - - OSSEC HIDS v2.8 Installation Script - http://ossec.github.io/ - - You are about to start the installation process of the OSSEC HIDS. - You must have a C compiler pre-installed in your system. - If you have any questions or comments, please send an e-mail - to dcid@ossec.net (or daniel.cid@gmail.com). - - - System: Linux deft 3.18.3-x86_64-linode51 - - User: root - - Host: deft - - -- Press ENTER to continue or Ctrl-C to abort. -- - -4. Press **ENTER**. You should now see: - - 1- What kind of installation do you want (server, agent, local, hybrid or help)? - - You have these installation options: server, agent, local, or hybrid. - - ... - - 1- What kind of installation do you want (server, agent, local, hybrid or help)? local - -5. Choose **local**, which enables OSSEC to monitor the server it has been installed on. Select the defaults for subsequent options. Note that the email you specify has to be a valid one, because all OSSEC alerts will be sent to it. If installation is successful, you should see: - - - System is Debian (Ubuntu or derivative). - - Init script modified to start OSSEC HIDS during boot. - - - Configuration finished properly. - - - To start OSSEC HIDS: - /var/ossec/bin/ossec-control start - - - To stop OSSEC HIDS: - /var/ossec/bin/ossec-control stop - - - The configuration can be viewed or modified at /var/ossec/etc/ossec.conf - - Thanks for using the OSSEC HIDS. - If you have any question, suggestion or if you find any bug, - contact us at contact@ossec.net or using our public maillist at - ossec-list@ossec.net - ( http://www.ossec.net/main/support/ ). - - More information can be found at http://www.ossec.net - - --- Press ENTER to finish (maybe more information below). --- - - Press **ENTER** to finish. OSSEC is now installed, and ready to be configured. - -## Configure OSSEC - -OSSEC is chrooted to the `/var/ossec` directory, and its configuration file is `ossec.conf`, in the `/var/ossec/etc` directory. There are several modifications that you will need to make in `ossec.conf`, including: - -- Email Settings - -- Configure OSSEC to alert on new files - -- Directories to monitor - -- Files and directories to ignore - -- Active response block times - -- Rules for new files - - -Use the `sudo su` command to gain root access to the directory: - - sudo su - -Switch to the `/var/ossec/etc` directory. - -### Customize Email Settings - -Although you specified an email and OSSEC auto-discovered the SMTP server, there are a few changes that can be made to the email settings. - -1. Open `ossec.conf`. The email settings are at the top of the file: - - {{< file "ossec.conf" >}} - - yes - loginName@example.com - mail.example.com. - ossecm@hostname - - -{{< /file >}} - - - The `< email_to >` entry is the email specified during installation. That is the address in which OSSEC will send alerts, and can be changed at any time. The `< email_from >` entry is where OSSEC's alerts will appear to be sent from. If you do not change the default, some SMTP servers will mark the email alerts as spam. - -2. Modify the `< email_from >` line: - - {{< file "ossec.conf" >}} - - yes - loginName@example.com - mail.example.com. - loginName@example.com - - -{{< /file >}} - - - {{< note respectIndent=false >}} -The `< email_to >` and `< email_from >` values can be the same. If you are running your own mail server and it's on the same server that OSSEC is installed, you may change the `< smtp_server >` value to `localhost`. -{{< /note >}} - - After making changes to the email settings, save and close the file. - -3. Start OSSEC: - - /var/ossec/bin/ossec-control start - - If the current working directory is `/var/ossec/etc`, you can use: - - ../bin/ossec-control start - -4. After OSSEC has started, check your inbox for the first alert from OSSEC: - - OSSEC HIDS Notification. - 2015 Jan 30 08:22:52 - - Received From: deft->ossec-monitord - Rule: 502 fired (level 3) -> "Ossec server started." - Portion of the log(s): - - ossec: Ossec started. - - --END OF NOTIFICATION - - If the email is received, then the settings are working and subsequent alerts will also hit your inbox. - - -### Configure OSSEC to Alert on New Files - -By default OSSEC will not send out an alert when a new file is added to the system. - -1. Open `ossec.conf` and scroll down to the following section: - - {{< file "ossec.conf" >}} - - - 79200 - -{{< /file >}} - - -2. Modify the file: - - {{< file "ossec.conf" >}} - - - 79200 - - yes - -{{< /file >}} - - - Note that the `< frequency >` is the interval that OSSEC performs a system check. During testing, you may reduce that setting to a lower number, like 900. Afterwards, it can be changed back to the default. - - -### Modify Directories to Monitor - -1. Open `ossec.conf`. Below the `< frequency >` setting is a list of system directories that OSSEC has been configured to monitor: - - {{< file "ossec.conf" >}} - -/etc,/usr/bin,/usr/sbin -/bin,/sbin - -{{< /file >}} - - -2. OSSEC can check the home directory and, if hosting a website on the server, monitor the website's data directory. For the specified directories, OSSEC can be configured to report changes in real-time: - - {{< file "ossec.conf" >}} - -/etc,/usr/bin,/usr/sbin -/bin,/sbin -/home/username,/var/www - -{{< /file >}} - - -3. Another directory you may want to configure OSSEC to monitor is `/var/ossec`, OSSEC's installation directory. However, if monitored, OSSEC **must** be configured to ignore certain directories within `/var/ossec`, otherwise you risk running out of disk space. - -### Modify Files and Directories to Ignore - -For a Linux system, the default list of files and directories to ignore are: - -{{< file "ossec.conf" >}} - -/etc/mtab -/etc/mnttab -/etc/hosts.deny -/etc/mail/statistics -/etc/random-seed -/etc/adjtime -/etc/httpd/logs -/etc/utmpx -/etc/wtmpx -/etc/cups/certs -/etc/dumpdates -/etc/svc/volatile - -{{< /file >}} - - -If OSSEC is configured to monitor `/var/ossec`, include the additional lines: - -{{< file "ossec.conf" >}} -/var/ossec/logs -/var/ossec/queue -/var/ossec/var -/var/ossec/tmp -/var/ossec/stats - -{{< /file >}} - - -### Modify Active Response Block Times - -OSSEC uses tcpwrappers (`host.deny`) and iptables to ban any IP addresses that trigger an alert. By default, such IP addresses are blocked for 600 seconds, or 10 minutes. This can be changed to a longer timeframe. - -To alter the timeframe of a ban, change the `< timeout >` setting to suit. Note that it has to be in **seconds**: - -{{< file "ossec.conf" >}} - - - - host-deny - local - 6 - 600 - - - - - - firewall-drop - local - 6 - 600 - - -{{< /file >}} - - -After making changes, save and close the file. - -### Modify Rule for New Files - -1. Move to the `/var/ossec/rules` directory: - - cd /var/ossec/rules - - The rules are located in a series of XML files and all rule definitions are found in `ossec_rules.xml`, which should not be modified. All rule modifications must be made in `local_rules.xml`. - -2. The rule that fires on new files is rule number **554**. The chunk of code that defines that rule in `ossec_rules.xml` is: - - {{< file "ossec_rules.xml" >}} - - ossec - syscheck_new_entry - File added to the system. - syscheck, - - -{{< /file >}} - - -3. Since OSSEC does not alert on rules that are **level 0**, that rule has to be modified in `local_rules.xml` so that OSSEC can fire when a new file is added to the system. The rule modification should be located between the `< group > ... < /group >` tags: - - {{< file "local_rules.xml" >}} - - ossec - syscheck_new_entry - File added to the system. - syscheck, - - - - - - - -{{< /file >}} - - -5. Save and close the file. - -## Restarting OSSEC - -After any modification is made to OSSEC, it has to be restarted: - - /var/ossec/bin/ossec-control restart - -If it restarts with no errors, it will output the following: - - Killing ossec-monitord .. - Killing ossec-logcollector .. - Killing ossec-syscheckd .. - Killing ossec-analysisd .. - Killing ossec-maild .. - Killing ossec-execd .. - OSSEC HIDS v2.8 Stopped - Starting OSSEC HIDS v2.8 (by Trend Micro Inc.)... - Started ossec-maild... - Started ossec-execd... - Started ossec-analysisd... - Started ossec-logcollector... - Started ossec-syscheckd... - Started ossec-monitord... - Completed. - -Otherwise, you will receive a configuration error: - - Starting OSSEC HIDS v2.8 (by Trend Micro Inc.)... - ossec-analysisd: Configuration error. Exiting. - -If this happens, check OSSEC's error log. It is advised to check the error log even if OSSEC restarts successfully, because may contain other errors that don't prevent OSSEC from restarting, but may need your attention. The error log file is `ossec.log` under the `/var/ossec/logs` directory. - -## What to Expect - -After making any final modifications and restarting OSSEC, you should receive another alert saying that OSSEC has started. If any rule is triggered after that, you will also receive similar alerts in your inbox: - - OSSEC HIDS Notification. - 2015 Jan 30 09:57:06 - - Received From: deft->syscheck - Rule: 550 fired (level 7) -> "Integrity checksum changed." - Portion of the log(s): - - Integrity checksum changed for: '/home/finid/sidebar.php' - Size changed from '0' to '19' - What changed: - 0a1 - - this is a php file - - Old md5sum was: 'd41d8cd98f00b204e9800998ecf8427e' - New md5sum is : 'df4f22f84b5ceb2c31cbb391883f2c83' - Old sha1sum was: 'da39a3ee5e6b4b0d3255bfef95601890afd80709' - New sha1sum is : '297b920c943134074690344a3da83f19483a93ef' - - --END OF NOTIFICATION - ---- - - OSSEC HIDS Notification. - 2015 Jan 30 09:26:02 - - Received From: deft->netstat -tan |grep LISTEN |grep -v 127.0.0.1 | sort - Rule: 533 fired (level 7) -> "Listened ports status (netstat) changed (new port opened or closed)." - Portion of the log(s): - - ossec: output: 'netstat -tan |grep LISTEN |grep -v 127.0.0.1 | sort': - tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:43157 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN - tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 ::1:25 :::* LISTEN - tcp6 0 0 :::22 :::* LISTEN - tcp6 0 0 :::53262 :::* LISTEN - Previous output: - ossec: output: 'netstat -tan |grep LISTEN |grep -v 127.0.0.1 | sort': - tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:43157 0.0.0.0:* LISTEN - tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 ::1:25 :::* LISTEN - tcp6 0 0 :::22 :::* LISTEN - tcp6 0 0 :::53262 :::* LISTEN - - - --END OF NOTIFICATION - ---- - - OSSEC HIDS Notification. - 2015 Jan 30 09:57:54 - - Received From: deft->syscheck - Rule: 553 fired (level 7) -> "File deleted. Unable to retrieve checksum." - Portion of the log(s): - - File '/home/user5/footer.php' was deleted. Unable to retrieve checksum. - - --END OF NOTIFICATION - ---- - - OSSEC HIDS Notification. - 2015 Jan 30 09:19:16 - - Received From: deft->/var/log/dpkg.log - Rule: 2902 fired (level 7) -> "New dpkg (Debian Package) installed." - Portion of the log(s): - - 2015-01-30 09:19:14 status installed nginx-common:all 1.2.1-2.2+wheezy3 - - - --END OF NOTIFICATION diff --git a/docs/guides/web-servers/apache/host-your-own-rss-reader-with-tiny-tiny-rss-on-centos-7/index.md b/docs/guides/web-servers/apache/host-your-own-rss-reader-with-tiny-tiny-rss-on-centos-7/index.md deleted file mode 100644 index 1e383eb5e80..00000000000 --- a/docs/guides/web-servers/apache/host-your-own-rss-reader-with-tiny-tiny-rss-on-centos-7/index.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -slug: host-your-own-rss-reader-with-tiny-tiny-rss-on-centos-7 -title: 'Host Your Own RSS Reader with Tiny Tiny RSS on CentOS 7' -description: 'This article gives you step-by-step instructions on how you can self-host your own RSS reader on a Linode running CentOS 7 with the Tiny Tiny RSS app.' -authors: ["Tyler Langlois"] -contributors: ["Tyler Langlois"] -published: 2017-03-30 -keywords: ["apache", "centos", "rss", "reader", "ttrss", "tt-rss"] -tags: ["centos","web server","apache"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[Tiny Tiny RSS Installation Notes](https://tt-rss.org/gitlab/fox/tt-rss/wikis/InstallationNotes)' ---- - -[Tiny Tiny RSS](https://tt-rss.org/) (or tt-rss for short) is an open-source, self-hosted RSS reader that runs on PHP and a traditional SQL database. Running your own RSS aggregator puts you in control of your data, and Tiny Tiny RSS even supports [mobile apps](https://play.google.com/store/apps/details?id=org.ttrssreader) that connect to your server. - -![Install Tiny Tiny RSS on CentOS 7](host-your-own-rss-reader-with-tiny-tiny-rss-on-centos-7.png) - -This guide will walk through the steps necessary to install and configure Tiny Tiny RSS on a Linode running CentOS 7, using MariaDB as the database and Apache as the web server. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's hostname and timezone. - -2. Follow the steps in the [LAMP on CentOS 7](/cloud/guides/how-to-install-a-lamp-stack-on-centos-7) guide. - -3. Make sure your system is up to date: - - sudo yum update - -## Preparing MySQL - -1. Connect to your MariaDB database as the root user: - - mysql -u root -p - -2. From the MariaDB shell, issue the following commands to create a new database and user for Tiny Tiny RSS. Replace **MyPassword** with a strong password: - - create database ttrss; - grant all on ttrss.* to 'ttrss' identified by 'MyPassword'; - exit - -## Preparing Apache - -1. Ensure that necessary PHP prerequisites are installed: - - sudo yum install -y php-mysql php-mbstring php-intl - -2. Add a configuration file under `/etc/httpd/conf.d/ttrss.conf` to secure the directories that Tiny Tiny RSS will use: - - {{< file "/etc/httpd/conf.d/ttrss.conf" aconf >}} - - Require all denied - - - - - Require all denied - - - -{{< /file >}} - - -3. Restart Apache to apply your changes: - - sudo systemctl restart httpd - -## Install Tiny Tiny RSS - -The recommended installation method for Tiny Tiny RSS is to clone the repository with `git`, as this simplifies the update process. - -1. Install `git`: - - sudo yum install -y git - -2. Clone the codebase into `/var/www/html`: - - sudo git clone https://tt-rss.org/git/tt-rss.git /var/www/html - - {{< note respectIndent=false >}} -This command will clone tt-rss into the `/var/www/html` directory at the root, which means you will access the application at the root URL of your webserver (for example, at http://myserver). -If you would prefer to use Tiny Tiny RSS under a separate URL (for example, at http://myserver/tt-rss), you can change the directory indicated in the `git clone` command to `/var/www/html/tt-rss`. - -If you decide to use a different location, note that you'll need to replace instances of `/var/www/html` in your Apache `ttrss.conf` file with the directory of your choosing. -{{< /note >}} - -3. Restart Apache to ensure that your changes have been applied: - - sudo systemctl restart httpd - -## Configure Tiny Tiny RSS - -At this point the application should be accessible under Apache. As an example, if your Linode had the IP address of `1.2.3.4`, browsing to `http://1.2.3.4` should result in the following screen: - -![Tiny Tiny RSS Installation Page](tiny-tiny-rss-install-page.png) - -1. Fill in the fields with the appropriate information: - - * From the **Database type** dropdown, choose `MySQL`. - * Under **Username**, fill in `ttrss`. - * Under **Password**, fill in the password chosen when setting up the ttrss database in MariaDB/MySQL. - * Use `ttrss` as the Database name. - * The **Host name** and **Port** fields can be left blank. - * The **Tiny Tiny RSS URL** section should automatically populate with the correct IP address for your Linode. If you are accessing your Linode via a DNS name instead of an IP address, be sure update the field to reflect this. - -2. After filling in the fields, click **Test configuration** to perform a preliminary check of your setup. If everything is ready, click the **Initialize database** button. - - {{< note respectIndent=false >}} -Initializing the database will wipe all data in the `ttrss` database. -If you are installing over a previous installation, perform any backups as necessary. -{{< /note >}} - -3. After the application initializes the MariaDB database, a message should appear warning you that TinyRSS cannot update `config.php` because the parent directory is not writeable. This is a *good* thing because any potential vulnerabilities in the web application cannot write files to disk. In order to finish configuring the application, follow the instructions to copy the full contents of the text box beginning with `}} -}} - - - If you need to customize your Tiny Tiny RSS configuration further (for example, if you have an SMTP server that you wish to use in conjunction with Tiny Tiny RSS to email you with feed news), you should do so by editing `config.php` now. - -4. Before using Tiny Tiny RSS, a few directory permissions must be changed so that Apache can write to them. The following commands will change only the necessary directories that require additional permissions: - - cd /var/www/html - sudo chgrp -R apache cache lock feed-icons - sudo chmod -R g+w cache lock feed-icons - -5. This command must be used to permit Apache to write to these directories on CentOS 7 systems with SELinux enabled. If SELinux has been disabled (check the output of the `sestatus` command if unsure), this step is not necessary: - - sudo chcon -R unconfined_u:object_r:httpd_sys_rw_content_t:s0 cache feed-icons lock - -### Feed Updates - -Now that Tiny Tiny RSS is up and running, create a systemd unit to automate the updating of your RSS feed. Create a file under `/etc/systemd/system/ttrss-updater.service` and copy the following information into it: - -{{< file "/etc/systemd/system/ttrss-updater.service" ini >}} -[Unit] -Description=ttrss_backend -After=network.target mysql.service - -[Service] -User=apache -ExecStart=/var/www/html/update_daemon2.php - -[Install] -WantedBy=multi-user.target - -{{< /file >}} - - -Start the service, and enable it to start at boot: - - sudo systemctl enable --now ttrss-updater - -## Using the Application - -1. Browse to the URL of your server, which should render the Tiny Tiny RSS login page: - - ![Tiny Tiny RSS Login Page](tiny-tiny-rss-login.png) - -2. Log in with the username `admin` and default password `password`. - -3. Change the administrator password to something stronger by clicking the **Actions** button in the top-right corner of the Tiny Tiny RSS main page. Select **Preferences**, then click the **Users** tab that appears. Click on the `admin` user and you'll see the following screen, with the option to change the user password: - - ![Tiny Tiny RSS User Editor](ttrss-user-editor.png) - - Note that if you prefer to log in with a less privileged user, you can create additional users from the **Users** page, by clicking **Create user**. - -At this point you can begin using Tiny Tiny RSS to subscribe to and read feeds. To get started, click on the **Actions** button (click *Exit preferences* if you are still in the preferences panel), and click the **Subscribe to feed...** link. diff --git a/docs/guides/web-servers/caddy/how-to-install-and-configure-caddy-on-centos-8/index.md b/docs/guides/web-servers/caddy/how-to-install-and-configure-caddy-on-centos-8/index.md deleted file mode 100644 index 77d5135b002..00000000000 --- a/docs/guides/web-servers/caddy/how-to-install-and-configure-caddy-on-centos-8/index.md +++ /dev/null @@ -1,134 +0,0 @@ ---- -slug: how-to-install-and-configure-caddy-on-centos-8 -title: "Install and Configure the Caddy Web Server on CentOS 8" -title_meta: "How to Install and Configure the Caddy Web Server on CentOS 8" -description: "In this guide, you will install the Caddy web server on CentOS 8. You will also configure Caddy to serve your site's domain over HTTPS." -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-03-05 -keywords: ['web server','caddy','https','Caddyfile'] -tags: ["centos","web server"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -relations: - platform: - key: install-caddy-server - keywords: - - distribution: CentOS 8 -aliases: [] ---- - -[Caddy](https://caddyserver.com/) is a fast, open-source, and security-focused web server written in [Go](https://golang.org/). Caddy includes modern features such as support for virtual hosts, minification of static files, and HTTP/2. Caddy is also the first web-server that can obtain and renew SSL/TLS certificates automatically using [Let's Encrypt](https://letsencrypt.org/). - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's [hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and [timezone](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#set-the-timezone). - -1. Complete the sections of our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to [create a standard user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account), and [harden SSH access](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#harden-ssh-access). - -1. Register (purchase) your site's domain name and follow our [Manage DNS Records](https://techdocs.akamai.com/cloud-computing/docs/manage-dns-records) guide to point the domain to your Linode. - -1. Update your system: - - sudo yum update - -1. Install the SELinux core policy Python utilities. This gives you the ability to manage SELinux settings in a fine-grained way. - - sudo yum install -y policycoreutils-python-utils - -## Install Caddy - -1. Install the `dnf-command(cpor)` plugin and enable `caddy`: - - sudo dnf install 'dnf-command(copr)' - sudo dnf copr enable @caddy/caddy - -1. Install Caddy: - - sudo dnf install caddy - -1. To verify the installation of caddy type: - caddy version - An output similar to the following appears: - - v2.4.3 h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I= - -## Allow HTTP and HTTPS Connections - -Caddy serves websites using HTTP and HTTPS protocols, so you need to allow access to the ports 80, and 443. - - sudo firewall-cmd --permanent --zone=public --add-service=http - sudo firewall-cmd --permanent --zone=public --add-service=https - sudo firewall-cmd --reload - -## Add Web Content - -1. Set up a home directory, **web root**, for your website: - - sudo mkdir -p /var/www/html/example.com - -1. Use SELinux’s `chcon` command to change the file security context for web content: - - sudo chcon -t httpd_sys_content_t /var/www/example.com -R - sudo chcon -t httpd_sys_rw_content_t /var/www/example.com -R - -1. Create a test page: - - echo 'Caddy Test Page

Hello, World!

' > /var/www/html/example.com/index.html - -## Configure the Caddyfile - -Add your hostname and web root to the Caddy configuration. Use an editor of your choice and replace `:80` with your domain name. Set the root directory of the site to `/var/www/html/example.com` Replace `example.com` with your site's domain name: - -{{< file "/etc/caddy/Caddyfile" caddy >}} -example.com { - root * /var/www/html/example.com - file_server -} -{{< /file >}} - -## Start and Enable the Caddy Service - -1. Temporarily set SELinux to permissive mode in order to start the Caddy service. - - sudo setenforce 0 - -1. Enable the Caddy service: - - sudo systemctl start caddy - -1. Verify that the service is active: - - sudo systemctl status caddy - - An output similar to the following appears: - - {{< output >}} -● caddy.service - Caddy - Loaded: loaded (/usr/lib/systemd/system/caddy.service; disabled; vendor preset: disabled) - Active: active (running) since Thu 2021-09-02 18:25:29 IST; 4s ago - Docs: https://caddyserver.com/docs/ - Main PID: 19314 (caddy) - CGroup: /system.slice/caddy.service - └─19314 /usr/bin/caddy run --environ --config /etc/caddy/Caddyfile... - -Sep 02 18:25:29 caddy caddy[19314]: SHELL=/sbin/nologin -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1270738..."} -Sep 02 18:25:29 caddy systemd[1]: Started Caddy. -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1316314...]} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1317837...0} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1324193..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1324632..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1325648..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1326034..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1326299..."} -Hint: Some lines were ellipsized, use -l to show in full. - {{}} - - To check the latest logs without truncation use `sudo journalctl -u caddy --no-pager | less +G`. - -1. Set SELinux back to enforcing mode once you have successfully started the Caddy service. - - sudo setenforce 1 - -1. Type your domain in the browser on your local machine and you should see the test page. If everything is configured correctly, you should see a green lock symbol in the URL bar, indicating that your connection is secure. - diff --git a/docs/guides/web-servers/caddy/how-to-install-and-configure-caddy-on-ubuntu-18-04/index.md b/docs/guides/web-servers/caddy/how-to-install-and-configure-caddy-on-ubuntu-18-04/index.md deleted file mode 100644 index fa4c72c1bcc..00000000000 --- a/docs/guides/web-servers/caddy/how-to-install-and-configure-caddy-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -slug: how-to-install-and-configure-caddy-on-ubuntu-18-04 -title: "Install and Configure the Caddy Web Server on Ubuntu 18.04" -title_meta: "Install and Configure the Caddy Web Server on Ubuntu" -description: "In this guide, you will install the Caddy web server on Ubuntu 18.04. You will also configure Caddy to serve your site's domain over HTTPS." -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-03-05 -modified: 2022-02-04 -keywords: ['web server','caddy','https','Caddyfile'] -tags: ["web server","ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: CaddyWebServ_Ubuntu1804.png -relations: - platform: - key: install-caddy-server - keywords: - - distribution: Ubuntu 18.04 -aliases: [] ---- - -[Caddy](https://caddyserver.com/) is a fast, open-source, and security-focused web server written in [Go](https://golang.org/). Caddy includes modern features such as support for virtual hosts, minification of static files, and HTTP/2. Caddy is also the first web-server that can obtain and renew SSL/TLS certificates automatically using [Let's Encrypt](https://letsencrypt.org/). - -## Before You Begin - -1. Familiarize yourself with the [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's [hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and [timezone](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#set-the-timezone). - -1. Complete the sections of the [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to [create a standard user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account) and [harden SSH access](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#harden-ssh-access). - -1. Register (purchase) your site's domain name and follow our [Manage DNS Records](https://techdocs.akamai.com/cloud-computing/docs/manage-dns-records) guide to point the domain to your Linode. - -1. Update your system: - - sudo apt update && sudo apt upgrade - -## Install Caddy - -1. Download Caddy: - - sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https - curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg - curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list - -1. Install Caddy: - - sudo apt update && sudo apt install caddy - -1. To verify the installation of Caddy, run the following command: - - caddy version - - This should output a message similar to the text below: - - {{}} -v2.4.3 h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I= -{{}} - -## Allow HTTP and HTTPS Connections - -Caddy serves websites using HTTP and HTTPS protocols, so you need to allow access to the ports 80, and 443. - - sudo ufw allow proto tcp from any to any port 80,443 - -An output similar to the following appears: - -{{< output >}} -Rule added -Rule added (v6) -{{< /output >}} - -1. Verify the changes: - - sudo ufw status - -An output similar to the following appears: -{{< output >}} -Status: active - -To Action From --- ------ ---- -OpenSSH ALLOW Anywhere -80,443/tcp ALLOW Anywhere -OpenSSH (v6) ALLOW Anywhere (v6) -80,443/tcp (v6) ALLOW Anywhere (v6) -{{< /output >}} - -## Add Web Content - -1. Set up a home directory, **web root**, for your website: - - sudo mkdir -p /var/www/html/example.com - -1. Create a test page: - - echo 'Caddy Test Page

Hello, World!

' > /var/www/html/example.com/index.html - -## Configure the Caddyfile - -Add your hostname and web root to the Caddy configuration. Use an editor of your choice and replace `:80` with your domain name. Set the root directory of the site to `/var/www/html/example.com` Replace `example.com` with your site's domain name: - -{{< file "/etc/caddy/Caddyfile" caddy >}} -example.com { - root * /var/www/html/example.com - file_server -} -{{< /file >}} - -## Start and Enable the Caddy Service - -1. Enable the Caddy service: - - sudo systemctl start caddy - -1. Verify that the service is active: - - sudo systemctl status caddy - - An output similar to the following appears: - - {{< output >}} -● caddy.service - Caddy - Loaded: loaded (/usr/lib/systemd/system/caddy.service; disabled; vendor preset: disabled) - Active: active (running) since Thu 2021-09-02 18:25:29 IST; 4s ago - Docs: https://caddyserver.com/docs/ - Main PID: 19314 (caddy) - CGroup: /system.slice/caddy.service - └─19314 /usr/bin/caddy run --environ --config /etc/caddy/Caddyfile... - -Sep 02 18:25:29 caddy caddy[19314]: SHELL=/sbin/nologin -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1270738..."} -Sep 02 18:25:29 caddy systemd[1]: Started Caddy. -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1316314...]} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1317837...0} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1324193..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1324632..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1325648..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1326034..."} -Sep 02 18:25:29 caddy caddy[19314]: {"level":"info","ts":1630587329.1326299..."} -Hint: Some lines were ellipsized, use -l to show in full. - {{}} - -To check the latest logs without truncation use `sudo journalctl -u caddy --no-pager | less +G`. - -1. Open a web browser and visit your domain. You should see the contents of the `index.html`page that you created in the [Add Web Content section](#add-web-content). diff --git a/docs/guides/web-servers/lamp/how-to-install-a-lamp-stack-on-arch-linux/index.md b/docs/guides/web-servers/lamp/how-to-install-a-lamp-stack-on-arch-linux/index.md deleted file mode 100644 index 34d4c6d345c..00000000000 --- a/docs/guides/web-servers/lamp/how-to-install-a-lamp-stack-on-arch-linux/index.md +++ /dev/null @@ -1,246 +0,0 @@ ---- -slug: how-to-install-a-lamp-stack-on-arch-linux -title: How to Install a LAMP Stack on Arch Linux -description: 'A simple tutorial on installing a LAMP (Linux, Apache, MySQL, PHP) stack on an Arch Linux-powered server.' -authors: ["Alex Fornuto"] -contributors: ["Alex Fornuto"] -published: 2013-10-07 -modified: 2015-12-07 -keywords: ["arch lamp", "arch lamp stack", "lamp linux", "arch linode", "arch linux lamp", "arch linux", "arch", "lamp", "lamp stack", "apache", "mysql", "php"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[Arch Linux Wiki](http://wiki.ArchLinux.org/)' - - '[Apache HTTP Server Documentation](http://httpd.apache.org/docs/2.4/)' - - '[MySQL Documentation](http://dev.mysql.com/doc/)' - - '[Oracle MySQL and MariaDB Comparison](https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility/)' - - '[PHP Documentation](http://www.php.net/docs.php)' -relations: - platform: - key: install-lamp-stack - keywords: - - distribution: Arch Linux -tags: ["web server","php","mysql","apache","lamp"] ---- - -A LAMP (Linux, Apache, MySQL, PHP) stack is a common web stack used to prepare servers for hosting web content. This guide shows you how to install a LAMP stack an Arch Linux server. - -Since Arch does not come in specific versions, this guide is up-to-date as of the December 2015 Arch update. - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Before You Begin - -1. Ensure that you have followed the [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guides, and the Linode's [hostname is set](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname). - -2. Update your system: - - sudo pacman -Syu - -## Apache - -### Install and Configure - -1. Install Apache 2.4: - - sudo pacman -Syu apache - -2. Edit the `httpd-mpm.conf` Apache configuration file in `/etc/httpd/conf/extra/` to adjust the resource use settings. The settings shown below are a good starting point for a **Linode 2GB**: - - {{< note respectIndent=false >}} -Before changing any configuration files, it is advised that you make a backup of the file. To make a backup: - -cp /etc/httpd/conf/extra/httpd-mpm.conf ~/httpd-mpm.conf.backup -{{< /note >}} - - {{< file "/etc/httpd/conf/extra/httpd-mpm.conf" aconf >}} - - StartServers 4 - MinSpareServers 20 - MaxSpareServers 40 - MaxRequestWorkers 200 - MaxConnectionsPerChild 4500 - - -{{< /file >}} - - -3. Edit the `httpd-default.conf` file to turn KeepAlive off. - - {{< file "/etc/httpd/conf/extra/httpd-default.conf" aconf >}} -KeepAlive Off - -{{< /file >}} - - -4. Set Apache to start at boot: - - sudo systemctl enable httpd.service - -### Add Name-Based Virtual Hosts - -Virtual hosting can be configured so that multiple domains (or subdomains) can be hosted on the server. These websites can be controlled by different users, or by a single user, as you prefer. There are different ways to set up virtual hosts; however, we recommend the method below. - -1. Open `httpd.conf` and edit the line `DocumentRoot /srv/http` to define the default document root: - - {{< file "/etc/httpd/conf/httpd.conf" >}} -DocumentRoot "/srv/http/default" - -{{< /file >}} - - -2. Uncomment the line that reads `Include conf/extra/httpd-vhosts.conf` near the end of the `/etc/httpd/conf/httpd.conf` file: - - {{< file "/etc/httpd/conf/httpd.conf" apache >}} -Include conf/extra/httpd-vhosts.conf - -{{< /file >}} - - -2. Open `httpd-vhosts.conf`, under the `extra` folder. Edit the example virtual hosts block to resemble the ones below, replacing `example.com` with your domain. - - {{< file "/etc/httpd/conf/extra/httpd-vhosts.conf" aconf >}} - - ServerAdmin webmaster@example.com - ServerName example.com - ServerAlias www.example.com - DocumentRoot /srv/http/example.com/public_html/ - ErrorLog /srv/http/example.com/logs/error.log - CustomLog /srv/http/example.com/logs/access.log combined - - Order deny,allow - Allow from all - - - -{{< /file >}} - - - Remove the second example in the file, or use it configure a second website. - - {{< note respectIndent=false >}} -`ErrorLog` and `CustomLog` entries are suggested for more fine-grained logging, but are not required. If they are defined (as shown above), the `logs` directories must be created before you restart Apache. -{{< /note >}} - -4. Create the directories referenced in the configuration above: - - sudo mkdir -p /srv/http/default - sudo mkdir -p /srv/http/example.com/public_html - sudo mkdir -p /srv/http/example.com/logs - -5. After you've set up your virtual hosts, issue the following command to run Apache for the first time: - - sudo systemctl start httpd.service - - You should now be able to access your website. If no files are uploaded you will see an *Index of /* page. - - {{< note respectIndent=false >}} -Should any additional changes be made to a configuration file restart Apache: - -sudo systemctl restart httpd.service -{{< /note >}} - -## MariaDB - -### Install and Configure - -By default, Arch Linux provides MariaDB as a relational database solution. MariaDB is an open source drop-in replacement for MySQL, and all system commands that reference `mysql` are compatible with it. - -1. Install the `mariadb`, `mariadb-clients` and `libmariadbclient` packages: - - sudo pacman -Syu mariadb mariadb-clients libmariadbclient - -2. Install the MariaDB data directory: - - sudo mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql - -3. Start MariaDB and set it to run at boot: - - sudo systemctl start mysqld.service - sudo systemctl enable mysqld.service - -4. Run `mysql_secure_installation`, a program that helps secure MySQL and MariaDB. `mysql_secure_installation` gives you the option to set your root password, disable root logins from outside localhost, remove anonymous user accounts, remove the test database and then reload the privilege tables: - - mysql_secure_installation - -### Create a Database - -1. Log into MariaDB: - - mysql -u root -p - - `-u ` specifies the user, and `-p` will prompt you for the password. - -2. You will see the MariaDB prompt. Create a database and create and grant a user permissions on the database: - - CREATE DATABASE webdata; - GRANT ALL ON webdata.* TO 'webuser' IDENTIFIED BY 'password'; - - In this example `webdata` is the name of the database, `webuser` is the username, and `password` is the user's password. Note that database usernames and passwords do not correlate to system user accounts. - -3. Quit MariaDB: - - quit - -With Apache and MariaDB installed, you are now ready to move on to installing PHP to provide scripting support for your web application. - -## PHP - -PHP makes it possible to produce dynamic and interactive pages using your own scripts and popular web development frameworks. Many popular web applications like WordPress are written in PHP. If you want to develop your websites using PHP, you must first install it. - -1. Install PHP: - - sudo pacman -Syu php php-apache - -2. Edit `/etc/php/php.ini` for better error messages and logs, and upgraded performance. These modifications provide a good starting point for a **Linode 2GB**: - - {{< file "/etc/php/php.ini" ini >}} -error_reporting = E_COMPILE_ERROR|E_RECOVERABLE_ERROR|E_ERROR|E_CORE_ERROR -log_errors = On -error_log = /var/log/php/error.log -max_input_time = 30 -extension=mysql.so - -{{< /file >}} - - - {{< note respectIndent=false >}} -Ensure that all lines noted above are uncommented. A commented line begins with a semicolon (**;**). -{{< /note >}} - -3. Create the log directory for PHP and give the Apache user ownership: - - sudo mkdir /var/log/php - sudo chown http /var/log/php - -4. Enable the PHP module in the `/etc/httpd/conf/httpd.conf` file by adding the following lines in the appropriate sections: - - {{< file "/etc/httpd/conf/httpd.conf" aconf >}} -# Dynamic Shared Object (DSO) Support -LoadModule php7_module modules/libphp7.so -AddHandler php7-script php - -# Supplemental configuration -# PHP 7 -Include conf/extra/php7_module.conf - -# Located in the -AddType application/x-httpd-php .php -AddType application/x-httpd-php-source .phps - -{{< /file >}} - -5. In the same file, comment out the line `LoadModule mpm_event_module modules/mod_mpm_event.so` by adding a `#` in front, and add the line `LoadModule mpm_prefork_module modules/mod_mpm_prefork.so`: - - {{< file "/etc/httpd/conf/httpd.conf" apache >}} -#LoadModule mpm_event_module modules/mod_mpm_event.so -LoadModule mpm_prefork_module modules/mod_mpm_prefork.so - -{{< /file >}} - - -6. Restart the Apache: - - sudo systemctl restart httpd.service diff --git a/docs/guides/web-servers/nginx/build-nginx-with-pagespeed-from-source/index.md b/docs/guides/web-servers/nginx/build-nginx-with-pagespeed-from-source/index.md deleted file mode 100644 index 725c32dc2b4..00000000000 --- a/docs/guides/web-servers/nginx/build-nginx-with-pagespeed-from-source/index.md +++ /dev/null @@ -1,336 +0,0 @@ ---- -slug: build-nginx-with-pagespeed-from-source -title: 'Build NGINX with PageSpeed From Source' -description: 'This guide provides you with step-by-step instructions for compiling the Nginx web server from scratch so you can use Google''s PageSpeed moduleon your website.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2019-02-01 -modified: 2018-02-12 -keywords: ["nginx","pagespeed","optimization"] -license: '[CC BY-ND 4.0](http://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -dedicated_cpu_link: true -tags: ["web server","nginx"] ---- - -![Build NGINX with PageSpeed From Source](build-nginx-pagespeed-from-source-title.jpg "Build NGINX with PageSpeed From Source title graphic") - -## What is Google PageSpeed? - -[PageSpeed](https://www.modpagespeed.com/) is a set of modules for NGINX and Apache which optimize and measure page performance of websites. Optimization is done by minifying static assets such as CSS and JavaScript, which decreases page load time. [PageSpeed Insights](https://developers.google.com/speed/pagespeed/insights/) is a tool that measures your site's performance, and makes recommendations for further modifications based on the results. - -There are currently two ways to get PageSpeed and NGINX working together: - -- Compile NGINX with support for PageSpeed, then compile PageSpeed. -- Compile PageSpeed as a [dynamic module](https://www.nginx.com/blog/compiling-dynamic-modules-nginx-plus/) to use with NGINX, whether NGINX was installed from source or a binary. - - {{< note respectIndent=false >}} -Installing NGINX from source requires several manual installation steps and will require manual maintenance when performing tasks like version upgrades. To install NGINX using a package manager see the [NGINX](/cloud/guides/web-servers/nginx) section. - {{< /note >}} - -This guide will show how to compile both NGINX and PageSpeed. If you would prefer to use PageSpeed as a module for NGINX, see [this NGINX blog post](https://www.nginx.com/blog/optimize-website-google-pagespeed-dynamic-module-nginx-plus/) for instructions. - - -## Before You Begin - -- You should not have a pre-existing installation of NGINX. If you do, back up the configuration files if you want to retain their information, and then purge NGINX. - -- You will need root access to the system, or a user account with `sudo` privileges. - -- Set your system's [hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname). - -- Update your system's packages. - - -## Considerations for a Self-Compiled NGINX Installation - -**Filesystem Locations**: When you compile NGINX from source, the entire installation, including configuration files, is located at `/usr/local/nginx/nginx/`. This is in contrast to an installation from a package manager, which places its configuration files in `/etc/nginx/`. - -**Built-in Modules**: When you compile NGINX from source, no additional modules are included unless explicitly specified, which means that HTTPS is not supported by default. Below you can see the output of `nginx -V` using the PageSpeed automated install command on Ubuntu 18.04 with no additional modules or options specified. - -{{< output >}} -root@localhost:~# /usr/local/nginx/sbin/nginx -V -nginx version: nginx/1.17.3 -built by gcc 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1) -configure arguments: --add-module=/root/incubator-pagespeed-ngx-latest-stable -{{< /output >}} - -Contrast this output with the same command run on the same Ubuntu system but with the binary installed from NGINX's repository: - -{{< output >}} -root@localhost:~# nginx -V -nginx version: nginx/1.17.3 -built by gcc 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1) -built with OpenSSL 1.1.1 11 Sep 2018 -TLS SNI support enabled -configure arguments: --add-module=/root/incubator-pagespeed-ngx-latest-stable --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_v2_module -{{< /output >}} - - -## Build NGINX and PageSpeed - -The official [PageSpeed documentation](https://www.modpagespeed.com/doc/build_ngx_pagespeed_from_source) provides a bash script to automate the installation process. - -{{< note >}} -The automated installation script will install several compilation tools needed to install PageSpeed. If you are using a production environment, ensure you uninstall any packages that are no longer needed after the installation has completed. -{{< /note >}} - -1. If you plan to serve your website using TLS, install the SSL libraries needed to compile the HTTPS module for NGINX: - - **CentOS/Fedora** - - sudo yum install openssl-devel - - **Ubuntu/Debian** - - sudo apt install libssl-dev - -2. Run the [Automated Install](https://www.modpagespeed.com/doc/build_ngx_pagespeed_from_source) bash command to start the installation: - - bash <(curl -f -L -sS https://ngxpagespeed.com/install) \ - --nginx-version latest - -3. During the build process, you'll be asked if you want to build NGINX with any additional modules. The PageSpeed module is already included, so you don't need to add it here. - - The options below are a recommended starting point; you can also add more specialized options for your particular use case. These options retain the directory paths, user and group names of pre-built NGINX binaries, and enable the SSL and HTTP/2 modules for HTTPS connections: - - --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_v2_module - -4. You'll be asked if you want to configure nginx with the options that you added, answer *Y* to continue. - {{< output >}} - ./configure --add-module=/home/sudouser/incubator-pagespeed-ngx-latest-stable --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_v2_module -Does this look right? [Y/n] y - -{{< /output >}} - -5. Next you'll be asked if you want to build NGINX. You'll be shown the destination directories for logs, configuration files and binaries. If these look correct, answer *Y* to continue. - - {{< output >}} -Configuration summary - + using system PCRE library - + using system OpenSSL library - + using system zlib library - - nginx path prefix: "/etc/nginx" - nginx binary file: "/usr/sbin/nginx" - nginx modules path: "/usr/lib/nginx/modules" - nginx configuration prefix: "/etc/nginx" - nginx configuration file: "/etc/nginx/nginx.conf" - nginx pid file: "/var/run/nginx.pid" - nginx error log file: "/var/log/nginx/error.log" - nginx http access log file: "/var/log/nginx/access.log" - nginx http client request body temporary files: "/var/cache/nginx/client_temp" - nginx http proxy temporary files: "/var/cache/nginx/proxy_temp" - nginx http fastcgi temporary files: "/var/cache/nginx/fastcgi_temp" - nginx http uwsgi temporary files: "/var/cache/nginx/uwsgi_temp" - nginx http scgi temporary files: "/var/cache/nginx/scgi_temp" - -Build nginx? [Y/n] - -{{< /output >}} - -5. If the build was successful, you'll see the following message: - - {{< output >}} -Nginx installed with ngx_pagespeed support compiled-in. - -If this is a new installation you probably need an init script to -manage starting and stopping the nginx service. See: - http://wiki.nginx.org/InitScripts - -You'll also need to configure ngx_pagespeed if you haven't yet: - https://developers.google.com/speed/pagespeed/module/configuration - -{{< /output >}} - -6. When you want to update NGINX, back up your configuration files and repeat steps two through four above to build with the new source version. - - -## Control NGINX - -NGINX can be controlled either by creating a systemd service or by calling the binary directly. Choose one of these methods and do not mix them. If you start NGINX using the binary commands, for example, systemd will not be aware of the process and will try to start another NGINX instance if you run `systemctl start nginx`, which will fail. - -**systemd** - -1. In a text editor, create `/lib/systemd/system/nginx.service` and add the following unit file from [the NGINX wiki](https://www.nginx.com/resources/wiki/start/topics/examples/systemd/): - - {{< file "/lib/systemd/system/nginx.service" >}} -[Unit] -Description=The NGINX HTTP and reverse proxy server -After=syslog.target network.target remote-fs.target nss-lookup.target - -[Service] -Type=forking -PIDFile=/run/nginx.pid -ExecStartPre=/usr/sbin/nginx -t -ExecStart=/usr/sbin/nginx -ExecReload=/bin/kill -s HUP $MAINPID -ExecStop=/bin/kill -s QUIT $MAINPID -PrivateTmp=true - -[Install] -WantedBy=multi-user.target -{{< /file >}} - -2. Enable NGINX to start on boot and start the server: - - systemctl enable nginx - systemctl start nginx - -3. NGINX can now be controlled as with any other systemd-controlled process: - - systemctl stop nginx - systemctl restart nginx - systemctl status nginx - -**NGINX binary** - -You can use NGINX's binary to control the process directly without making a startup file for your init system. - -1. Start NGINX: - - /usr/sbin/nginx - -2. Reload the configuration: - - /usr/sbin/nginx -s reload - -3. Stop NGINX: - - /usr/sbin/nginx -s stop - - - -## Configuration - -### NGINX - -1. Since the compiled options specified above are different than the source's defaults, some additional configuration is necessary. Replace *example.com* in the following commands with your Linode's public IP address or domain name: - - sudo useradd --no-create-home nginx - sudo mkdir -p /var/cache/nginx/client_temp - sudo mkdir /etc/nginx/conf.d/ - sudo mkdir /var/www/ - sudo mkdir /var/www/example.com - sudo chown nginx:nginx /var/www/example.com - sudo mv /etc/nginx/nginx.conf.default /etc/nginx/nginx.conf.backup-default - -2. In NGINX terminology, a *Server Block* equates to a website (similar to the *Virtual Host* in Apache terminology). Each NGINX site's configuration should be in its own file with the name formatted as `example.com.conf`, located at `/etc/nginx/conf.d/`. - - If you followed this guide or our [Getting Started with NGINX](/cloud/guides/getting-started-with-nginx-part-1-installation-and-basic-setup) series, then your site's configuration will be in a `server` block in a file stored in `/etc/nginx/conf.d/`. If you do not have this setup, then you likely have the `server` block directly in `/etc/nginx/nginx.conf`. See *[Server Block Examples](https://www.nginx.com/resources/wiki/start/topics/examples/server_blocks/)* in the NGINX docs for more info. - - Create a configuration file for your site with a basic server block inside: - - {{< file "/etc/nginx/conf.d/example.com.conf" nginx >}} -server { - listen 80; - listen [::]:80; - server_name example.com www.example.com; - access_log logs/example.access.log main; - error_log logs/example.error error; - - root /var/www/example.com/; - -} -{{< /file >}} - -3. Ensure that the firewall allows access to the nginx service. If you configured the firewall with `ufw` then do the following: - - a. Check the application configurations that `ufw` is aware of, by typing: - - sudo ufw app list - - b. Enable `Nginx HTTP` from the list: - - sudo ufw allow 'Nginx HTTP' - - c. Verify the change by typing: - {{< output >}} - Status: active - -To Action From --- ------ ---- -OpenSSH ALLOW Anywhere -Nginx HTTP ALLOW Anywhere -OpenSSH (v6) ALLOW Anywhere (v6) -Nginx HTTP (v6) ALLOW Anywhere (v6) - {{< /output >}} - -4. Start NGINX: - - **systemd**: - - systemctl start nginx - - **Other init systems**: - - /usr/sbin/nginx - -5. Verify NGINX is working by going to your site's domain or IP address in a web browser. You should see the NGINX welcome page: - - ![NGINX welcome page](nginx-welcome.png "NGINX welcome page") - - -### PageSpeed - -1. Create PageSpeed's cache location and change its ownership to the `nginx` user and group: - - sudo mkdir /var/cache/ngx_pagespeed/ - sudo chown nginx:nginx /var/cache/ngx_pagespeed/ - -2. Add the PageSpeed directives to your site configuration's `server` block as shown below. - - {{< file "/etc/nginx/conf.d/example.com.conf" nginx >}} -server { - - ... - - pagespeed on; - pagespeed FileCachePath "/var/cache/ngx_pagespeed/"; - pagespeed RewriteLevel OptimizeForBandwidth; - - location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" { - add_header "" ""; - } - - location ~ "^/pagespeed_static/" { } - location ~ "^/ngx_pagespeed_beacon$" { } - - } -{{< /file >}} - - {{< note respectIndent=false >}} -`RewriteLevel OptimizeForBandwidth` is a [safer choice](https://www.modpagespeed.com/doc/optimize-for-bandwidth) than the default CoreFilters rewrite level. -{{< /note >}} - -3. NGINX supports HTTPS by default, so if your site already is set up with a TLS certificate, add the two directives below to your site's `server` block, pointing to the correct location [depending on your system](https://www.modpagespeed.com/doc/https_support#configuring_ssl_certificates). - - pagespeed SslCertDirectory directory; - pagespeed SslCertFile file; - -4. Reload your configuration: - - /usr/sbin/nginx/ -s reload - - Or restart nginx: - - systemctl restart nginx - -5. Test PageSpeed is running and NGINX is successfully serving pages. Substitute *example.com* in the cURL command with your Linode's domain name or IP address. - - curl -I -X GET example.com - - The output should be similar to below. If the response contains an HTTP 200 response and *X-Page-Speed* is listed in the header with the PageSpeed version number, everything is working correctly. - - {{< output >}} -HTTP/1.1 200 OK -Server: nginx/1.17.3 -Content-Type: text/html -Transfer-Encoding: chunked -Connection: keep-alive -Date: Sat, 07 Sep 2019 07:16:44 GMT -X-Page-Speed: 1.13.35.2-0 -Cache-Control: max-age=0, no-cache -{{< /output >}} - -6. Use [PageSpeed Insights](https://developers.google.com/speed/pagespeed/insights/) to test your site for additional improvement areas. diff --git a/docs/guides/web-servers/squid/squid-http-proxy-centos-8/index.md b/docs/guides/web-servers/squid/squid-http-proxy-centos-8/index.md deleted file mode 100644 index f58024cd2b7..00000000000 --- a/docs/guides/web-servers/squid/squid-http-proxy-centos-8/index.md +++ /dev/null @@ -1,238 +0,0 @@ ---- -slug: squid-http-proxy-centos-8 -title: 'Creating an HTTP Proxy Using Squid on CentOS 8' -title_meta: 'How to Create an HTTP Proxy Using Squid on CentOS 8' -description: This guide shows how to use Squid to create an HTTP proxy server on your Linode running CentOS 8. -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-04-14 -keywords: ["squid", "proxy", "centos", "8", "http"] -tags: ["proxy","web server","centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Creating_an_HTTP_Proxy_Using_Squid_on_Centos8_1200x631.png -external_resources: - - '[Squid Official Site](http://www.squid-cache.org/)' - - '[Configure Proxy on Windows](https://docs.microsoft.com/en-us/windows/security/threat-protection/microsoft-defender-atp/configure-proxy-internet)' - - '[Proxy Server Settings on macOS](https://support.apple.com/en-in/guide/mac-help/mchlp2591/mac)' - - '[Connection Settings in Firefox](https://support.mozilla.org/en-US/kb/connection-settings-firefox)' -relations: - platform: - key: install-squid-proxy - keywords: - - distribution: CentOS 8 -aliases: [] ---- - -This guide will show you how to create your own HTTP proxy using Squid, a highly customizable proxy/cache application, on CentOS 8. An HTTP proxy acts as an intermediary between you and the internet. While connected to your Squid HTTP proxy, you will be able to: - -- Anonymously access internet services. -- Bypass certain regional and local network restrictions. - -{{< note >}} -The traffic passed from your client to your Squid HTTP proxy will not be encrypted and will still be visible on your local network. If you are looking for a solution that offers greater security, you may want to look at our guides on [Setting up an SSH Tunnel](/cloud/guides/setting-up-an-ssh-tunnel-with-your-linode-for-safe-browsing) or [Deploy OpenVPN Access Server with Quick Deploy Apps](/cloud/marketplace-docs/guides/openvpn). -{{< /note >}} - -## Install Squid - -1. Secure your Linode by completing the instructions in our guide on [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance), including adding a limited user account and configuring a firewall. - - {{< note respectIndent=false >}} -This guide is written for a limited, non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you are not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -1. Ensure that your system is up-to-date: - - sudo yum update && sudo yum upgrade - -1. Install Squid using the `yum` software package manager: - - sudo yum install squid - -1. Copy the original configuration file to keep as a backup: - - sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.default - - {{< note respectIndent=false >}} -The Squid configuration file includes comprehensive documentation in its commented lines, along with several uncommented rules that will remain active. These default rules should not be modified while you are following this guide. To gain a deeper understanding of Squid's options and default settings, you can review the full configuration file. -{{< /note >}} - -## Configure Client Access - -Now that you have Squid installed on your Linode, you can configure ways for it to accept connections and serve as an HTTP proxy. The following sections provide different ways for your Squid HTTP proxy to authenticate client connections. You can configure Squid to use either or both authentication methods. - -### IP Address Authentication - -A simple way to use Squid as an HTTP proxy is to use a client's IP address for authentication. - -1. Edit the Squid configuration file and add the following lines at the beginning of the file: - - {{< file "/etc/squid/squid.conf" >}} -acl client src 192.0.2.0 # Home IP -http_access allow client -{{< /file >}} - - Replace `client` with a name that identifies the client computer that will connect to your Squid HTTP proxy, then replace `192.0.2.0` with the client computer's IP address. You can also update the optional comment `# Home IP` to further describe the client. - -1. Alternatively, you can configure multiple clients by adding new `acl` lines to `/etc/squid/squid.conf` and including them in the `http_access allow` line as follows: - - {{< file "/etc/squid/squid.conf" >}} -acl client1 src 192.0.2.0 # Home IP -acl client2 src 192.0.2.1 # Work IP -http_access allow client1 client2 -{{< /file >}} - - Replace `client1` and `client2` with names that identify the client computers, then replace `192.0.2.0` and `192.0.2.1` with their corresponding IP addresses. Update the optional comments `# Home IP` and `# Work IP` with accurate descriptions to help keep track of multiple clients. Access to the proxy is granted by adding the names defined by each `acl` to the `http_access allow` line. - -### User/Password Authentication - -You can also configure your Squid HTTP proxy to accept authentication with usernames and passwords. - -1. Install `htpasswd` by installing the Apache utility programs. If you have installed Apache on your Linode, you will already have it and can skip this step. - - sudo yum install httpd-tools - -1. Create a file to store Squid users and passwords: - - sudo touch /etc/squid/squid_passwd - -1. Change ownership of the password file: - - sudo chown squid /etc/squid/squid_passwd - -1. Create a username password pair, replacing `user1` with the name of the user you'd like to add: - - sudo htpasswd /etc/squid/squid_passwd user1 - - You will be prompted to create a password for this user: - - {{< output >}} -New password: -Re-type new password: -Adding password for user user1 -{{< /output >}} - - You can repeat this step at any time to create new users. - -1. Check the location of the `nsca_auth` file: - - sudo rpm -ql squid | grep ncsa_auth - -1. Edit the Squid configuration file and add the following lines at the beginning of the file: - - {{< note respectIndent=false >}} -Ensure that you update `/usr/lib64/squid/basic_ncsa_auth` below with the location of the `nsca_auth` file that you checked in the previous step. -{{< /note >}} - - {{< file "/etc/squid/squid.conf" >}} -auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/squid_passwd -acl ncsa_users proxy_auth REQUIRED -http_access allow ncsa_users -{{< /file >}} - -1. To remove a user's access to the proxy, you must delete the corresponding entry in the `squid_passwd` file. Each user is represented in the file on a single line in the format of `user:passwordhash`: - - {{< file "/etc/squid/squid_passwd" >}} -user1:\$p948w3nvq3489v6npq396g user2:\$q3cn478554387cq34n57vn -{{< /file >}} - - If you are using Nano, the command `Control+k` will remove the entire line where the cursor rests. - - Once you've saved and exited the file, complete user removal by restarting Squid: - - sudo systemctl restart squid - -### Combined Authentication - -You can combine authentication methods using the same `acl` definitions that you have added in the previous two sections by using a single `http_access` rule. - -1. Remove any previous `http_access` lines you have added. - -1. Edit the Squid configuration file so that the lines you have added at the beginning of the file follow this form: - - {{< file "/etc/squid/squid.conf" >}} -acl client1 src 192.0.2.0 # Home IP -acl client2 src 192.0.2.1 # Work IP -auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/squid_passwd -acl ncsa_users proxy_auth REQUIRED -http_access allow client1 client2 ncsa_users -{{< /file >}} - - {{< note respectIndent=false >}} -Take care to avoid using multiple `http_access` rules when combining authentication methods, as Squid will follow the rules in the order that they appear. By using a single `http_access` rule for your `acl` definitions, you will ensure that several authentication methods will apply to each client that attempts to connect to your Squid HTTP proxy. -{{< /note >}} - -## Anonymize Traffic - -Here, you will add rules to mask client IP addresses from the servers that receive traffic from you Squid HTTP proxy. Without these rules, the originating client IP addresses may be passed on through the `X-Forwarded For` HTTP header. - -Add the following lines at the beginning of the Squid configuration file: - -{{< file "/etc/squid/squid.conf" >}} -forwarded_for off -request_header_access Allow allow all -request_header_access Authorization allow all -request_header_access WWW-Authenticate allow all -request_header_access Proxy-Authorization allow all -request_header_access Proxy-Authenticate allow all -request_header_access Cache-Control allow all -request_header_access Content-Encoding allow all -request_header_access Content-Length allow all -request_header_access Content-Type allow all -request_header_access Date allow all -request_header_access Expires allow all -request_header_access Host allow all -request_header_access If-Modified-Since allow all -request_header_access Last-Modified allow all -request_header_access Location allow all -request_header_access Pragma allow all -request_header_access Accept allow all -request_header_access Accept-Charset allow all -request_header_access Accept-Encoding allow all -request_header_access Accept-Language allow all -request_header_access Content-Language allow all -request_header_access Mime-Version allow all -request_header_access Retry-After allow all -request_header_access Title allow all -request_header_access Connection allow all -request_header_access Proxy-Connection allow all -request_header_access User-Agent allow all -request_header_access Cookie allow all -request_header_access All deny all -{{< /file >}} - -## Enable Connections - -Next, you will enable clients to connect to your Squid HTTP proxy. - -1. Save and exit the Squid configuration file. - -1. Restart Squid to enable the rules you have added: - - sudo systemctl restart squid - -1. Implement firewall rules to enable port `3128`, which is the default service port used by Squid: - - sudo firewall-cmd --add-port=3128/tcp --permanent - sudo firewall-cmd --reload - - You can find more information on configuring firewall rules for CentOS in our guide on [Introduction to FirewallD on CentOS](/cloud/guides/introduction-to-firewalld-on-centos). - -## Connect to your Squid HTTP Proxy - -Your Squid HTTP proxy is now ready to accept client connections and anonymously handle internet traffic. - -At this point, you can configure your local browser or operating system's network settings to use your Linode as an HTTP proxy. The settings to do this will vary depending on your OS and browser. Instructions for certain OS and browser settings are located in the [More Information](#more-information) section below. - -Generally, connecting to your Squid HTTP proxy requires the following information: - -- The IP address or domain name associated with your Linode. -- The port that is being used by Squid. The default port is `3128`. -- A username and password if you have configured them for authentication. - -Once you have established your OS or browser settings, test the connection by pointing your browser at a website that tells you your IP address, such as: - -- [ifconfig.me](http://ifconfig.me) -- [WhatIsMyIP.com](http://www.whatismyip.com/) -- [Googling "what is my ip"](https://www.google.com/search?q=what+is+my+ip) - -The result should display your Linode's IP address instead of the IP address of your client computer. diff --git a/docs/guides/websites/cms/basics/cms-overview/index.md b/docs/guides/websites/cms/basics/cms-overview/index.md deleted file mode 100644 index b21b2e2cdcd..00000000000 --- a/docs/guides/websites/cms/basics/cms-overview/index.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -slug: cms-overview -title: "Content Management Systems: An Overview" -description: "An overview of the three content management systems that Linode supports" -authors: ["Linode"] -contributors: ["Linode"] -published: 2014-10-17 -modified: 2017-02-15 -keywords: ["drupal", "WordPress", "joomla", "cms", "content management system", "content management framwork"] -tags: ["drupal","wordpress","cms","lamp"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] ---- - -A content management system, or CMS, creates and manages websites. CMS applications eliminate the need for programming. Once running, they enable non-technical professionals to upload and administer their own content. Any coding is replaced with a user-friendly, point-and-click interface. There are dozens of popular CMS apps available, and many are free. Every CMS has a different design with particular web functionalities, aimed at specific users. CMS applications have been around since the late 1990s, and they continue to reach larger and larger audiences. - -![Content Management Systems: an Overview](content-management-systems-an-overview.png "Content Management Systems: an Overview") - -Even experienced programmers often choose using a CMS over coding a website. Content management systems foster quicker website development. A CMS website can be created and deployed within days. Many of the more popular CMS applications are open sourced. Open sourcing allows users to write helpful documentation or develop new add-ons. Through extremely active online communities, open sourcing creates plug-ins, modules, comments, questions, and technical answers. Help is always just a search away. - -## Content Management Systems on a Linode - -Linode has installation guides for [Drupal](/cloud/guides/managing-web-content-with-drupal-7), [WordPress](/cloud/guides/how-to-install-and-configure-wordpress), and [Joomla](/cloud/guides/manage-web-content-with-joomla). Together, they are the most popular CMS apps on the web. However, before you install, you'll need a running Linode, so see the [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guide. For security, work through the [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide. And, finally for server configuration, create a LAMP stack with the [Hosting a Website](/cloud/guides/hosting-a-website-ubuntu-18-04) guide. - -{{< note >}} -A LAMP stack is a System Admin term for a web server using Linux, Apache, MySQL, and PHP. Linux is an operating system, Apache is web-server software, MySQL is a database, and PHP is a programming language. This is a common stack combination and supports many CMS applications. -{{< /note >}} - -Using a Linode gives you more control and a greater understanding of website administration. Linux and content management systems play a huge role in the open-source world, and now, you can begin learning both. - -## Themes and Templates - -Most content management systems create their webpage layout with a theme. The appeal of any site starts with visual design. The appearance of your homepage keeps visitors coming back for more content. After installation, picking a theme is the next major decision. No worries, it's never too difficult to change. - -Themes, sometimes called skins, are HTML or CSS based. They act as templates, which means they are customizable. For example, a theme can create the header image divisions, column alignments, title text placement, webpage width, paragraph placement, side bars, and side bar modules. CMS applications may have default themes already available or other themes available on the web or for purchase. - -## Drupal, Joomla, and WordPress - -Three of the world's most popular content management systems are open source, free, and top quality. Each has its own strengths and development history. - -### Drupal - -[Drupal](/cloud/guides/managing-web-content-with-drupal-7) manages many visually stunning, popular sites like Weather.com, WhiteHouse.gov, and Rutgers.edu. Drupal has been in continuous release since 2001. Because of an active development cycle, Drupal releases an update every two to four months. A huge user community attends bi-annual conferences in North America and Europe. - -Drupal's strength lies in its organization. Drupal starts with the Drupal core, a base set of files, which allows for package add-ons like themes or modules. The core may have a basic set of modules or built-in themes, but can be continually customized. Optionally, a Drupal distribution is a similar to the core but centered around specific site needs. Distributions exist for kickstarter sites, news media sites, or community-based sites, to name a few. - -### Joomla - -[Joomla](/cloud/guides/manage-web-content-with-joomla) has been in release since 2005 and manages sites like eBay, General Electric, and Ikea. Joomla released version 4.0 in March 2015. While Drupal has more modules and themes, Joomla has more extensions with thousands of add-on options, although the idea of modules and extensions often overlap. Both Drupal and Joomla have users in the millions, which means thousands of user comments and help topics can be found online. - -Joomla works best for intermediate users, and while it lends itself to social collaboration or community-driven website design, it can be customized toward any situation. The installation process is uncomplicated, but generating content will not be as easy as it is with WordPress. - -### WordPress - -Originally built as a blogger platform, [WordPress](/cloud/guides/how-to-install-and-configure-wordpress) is arguably the easiest to use and the most popular. Initially released in 2003, it has grown rapidly since. The WP platform is used for over 60 million websites, and 100,000 new websites are created daily. - -WordPress is best for static content. However, WP sites are often built for complex, dynamic solutions because the community is so large and has added so many capabilities. The structure of the company behind WordPress, Automattic, and its philosophy represent the ability for large growth with such an open-source project. - -## Next Steps - -We have briefly covered the concepts of content management systems, CMS themes, LAMP stacks, CMS add-ons, open source, [Drupal](/cloud/guides/managing-web-content-with-drupal-7), [Joomla](/cloud/guides/manage-web-content-with-joomla), and [WordPress](/cloud/guides/how-to-install-and-configure-wordpress). Create a Linode, follow our installation guides, and start delivering your content to the world. - - - - - - - - - diff --git a/docs/guides/websites/cms/drupal/how-to-install-drupal-themes-and-modules-using-drush-on-centos-8/index.md b/docs/guides/websites/cms/drupal/how-to-install-drupal-themes-and-modules-using-drush-on-centos-8/index.md deleted file mode 100644 index b0efaf74291..00000000000 --- a/docs/guides/websites/cms/drupal/how-to-install-drupal-themes-and-modules-using-drush-on-centos-8/index.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -slug: how-to-install-drupal-themes-and-modules-using-drush-on-centos-8 -title: Install Drupal Themes and Modules Using Drush on CentOS 8 -title_meta: How to Install Drupal Themes and Modules Using Drush on CentOS 8 -description: 'Use Drush to install and enable themes and modules on your Drupal site running on CentOS 8.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2014-12-05 -modified: 2020-03-11 -keywords: ["drupal", "cms", "content management system", "content management framework", "centos", "drush"] -tags: ["drupal","centos","apache","lamp","php","cms"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: DrupalThemesMods_DrushCentOS8.png -external_resources: - - '[SSL Certificates](/cloud/guides/security/ssl)' - - '[Drush Commands](https://docs.drush.org/en/9.x/)' - - '[Backup and Migrate](https://www.drupal.org/docs/8/modules/backup-and-migrate/howto-for-backup-and-migrate)' -relations: - platform: - key: how-to-install-drupal-themes - keywords: - - distribution: CentOS 8 -aliases: [] ---- - -[Drush](https://www.drush.org/) is a command line tool for creating, administrating, and modifying Drupal websites. This tutorial uses Drush to install themes and modules. It will also briefly cover backups and migrations for Drupal websites. - -## Before You Begin - -Before installing themes, modules, and a backup system using Drush, make sure that the following prerequisites have been met: - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for [setting your Linode's hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and [timezone](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#set-the-timezone). - -1. Follow our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to [create a standard user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account), [harden SSH access](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#harden-ssh-access), and [create firewall rules](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-firewall) for your web server; you may need to make additional firewall exceptions for your specific application. - - {{% content "limited-user-note-shortguide" %}} - -1. Install and configure a [LAMP stack on CentOS 8](/cloud/guides/how-to-install-a-lamp-stack-on-centos-8) - -1. Install [Composer and Drush on CentOS 8](/cloud/guides/how-to-install-drush-on-centos-8) - -1. Make sure that your system is up to date, using: - - sudo yum update -{{< note >}} -The Drush commands to download or enable themes and modules vary depending on the version of Drush that you have installed. This guide uses Drush 10. -{{< /note >}} - -## Installing Themes with Drush - -In this section you will download, enable, and set a Drupal theme using Drush. - -1. Navigate to [Drupal's Download and Extend page](https://www.drupal.org/project/project_theme) to find the theme you want to download. The Drush download name is usually appended to the end of the Drupal theme page's URL; for example, `drupal.org/project/project/my_theme`. - -1. Move into your Drupal site's document root, `/var/www/html/example.com/public_html/`. Replace `example.com` with your own domain's name. - - cd /var/www/html/example.com/public_html - -1. Download your desired theme using [Composer](https://getcomposer.org/doc/). For example to download the `bootstrap` theme use: - - composer require drupal/bootstrap - - {{< note respectIndent=false >}} -If you receive an error related to not being able to write to the `composer.json` file, see the [Setting the Site’s Ownership and Permissions](/cloud/guides/how-to-install-drupal-using-drush-on-centos-8#setting-the-sites-ownership-and-permissions) section of the [Install Drupal using Drush on CentOS 8](/cloud/guides/how-to-install-drupal-using-drush-on-centos-8) guide. - -Ensure that your `/var/www/html/example.com/public_html` directory has user and group read, write, and execute permissions. - - sudo chmod 774 -R /var/www/html/example.com/public_html - {{< /note >}} - -1. Enable the theme that you downloaded in the previous step. For example, to enable the `bootstrap` theme run the following command. Replace `bootstrap` with the name of your theme. - - drush theme:enable bootstrap - -1. As an example, set the [Omega](https://www.drupal.org/project/omega) theme as the default and active theme: - - drush config-set system.theme default omega - - Check the homepage of your site and the new theme should appear. - -## Installing Modules with Drush - -Downloading and enabling a module is similar to working with a theme. However, modules can be used for almost any purpose. From enhancing public-facing functionality to providing a better administrative UI; there are thousands of Drupal modules. It is helpful to use modules with clear documentation, since once a module is installed, its interface can be challenging to use if it is not well documented and designed by the contributor. - -1. Move into your Drupal site's document root, `/var/www/html/example.com/public_html/`. Replace `example.com` with your own domain's name. - - cd /var/www/html/example.com/public_html - -1. Download the [Backup and Migrate](https://www.drupal.org/project/backup_migrate) module. This module enables you to perform the backup, restore, and migrate tasks - - composer require drupal/backup_migrate - -2. Enable the `backup_migrate` module: - - drush en backup_migrate -y - -3. Sign in to your Drupal site's browser interface and navigate to the **Extend** menu item (or **Administration > Extend**). Under the **OTHER** section, the Backup and Migrate module will be listed and selected. - -You have successfully installed and enabled a new module. The module is now running and ready to be used. - -## Backup a Drupal Site - -It's always important to keep regular backups of a website. Backups protect you from losing data due to configuration changes, vulnerabilities, or system failures. Backups should be stored on a separate system whenever possible. The **Backup and Migrate** module helps you to create backups of your site. - -1. To configure backup and migrate, navigate to **Administration > Configuration > Development > Backup and Migrate** - -2. For a quick backup, select the type of **Backup Source** and select the **Backup Destination**, and click **Backup now** -{{< note >}} -Always download a backup prior to updating or installing modules. -{{< /note >}} diff --git a/docs/guides/websites/cms/drupal/how-to-install-drupal-themes-and-modules-using-drush-on-ubuntu-18-04/index.md b/docs/guides/websites/cms/drupal/how-to-install-drupal-themes-and-modules-using-drush-on-ubuntu-18-04/index.md deleted file mode 100644 index 0fe6d429945..00000000000 --- a/docs/guides/websites/cms/drupal/how-to-install-drupal-themes-and-modules-using-drush-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -slug: how-to-install-drupal-themes-and-modules-using-drush-on-ubuntu-18-04 -title: Install Drupal Themes and Modules Using Drush on Ubuntu 18.04 -title_meta: How to Install Drupal Themes & Modules on Ubuntu 18.04 -description: 'Use Drush to install and enable themes and modules on your Drupal site running on Ubuntu 18.04.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2014-12-05 -modified: 2020-03-11 -keywords: ["drupal", "cms", "content management system", "content management framework", "ubuntu", "drush"] -tags: ["drupal","ubuntu","cms","lamp"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: DrupalThemesMods_DrushUbuntu1804.png -external_resources: - - '[SSL Certificates](/cloud/guides/security/ssl)' - - '[Drush Commands](https://docs.drush.org/en/9.x/)' - - '[Backup and Migrate](https://www.drupal.org/docs/8/modules/backup-and-migrate/howto-for-backup-and-migrate)' -relations: - platform: - key: how-to-install-drupal-themes - keywords: - - distribution: Ubuntu 18.04 -aliases: [] ---- - -Drush is a command line tool, which can be used for various Drupal projects. This tutorial uses Drush to install themes, modules, and covering some basic administration tasks such as backup and migrate for Drupal websites. - -Linode has another guide for installing Drush and creating a Drupal website, [Install Drupal using Drush on Ubuntu 18.04](/cloud/guides/how-to-install-drupal-using-drush-on-ubuntu-18-04). Depending on your experience level with Drush, you may want to start with that guide. - -## Before You Begin - -Before installing themes, modules, and a backup system with Drush, make sure that the following prerequisites have been met: - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for [setting your Linode's hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and [timezone](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#set-the-timezone). - -1. Follow our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to [create a standard user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account), [harden SSH access](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#harden-ssh-access), and [create firewall rules](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-firewall) for your web server; you may need to make additional firewall exceptions for your specific application. - - {{% content "limited-user-note-shortguide" %}} - -1. Install and configure a [LAMP stack on Ubuntu 18.04](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-18-04) - -1. Install [Composer and Drush on Ubuntu 18.04](/cloud/guides/how-to-install-drush-on-ubuntu-18-04) - - -1. Make sure that your system is up to date, using: - - sudo apt-get update && sudo apt-get upgrade -{{< note >}} -The Drush commands to download or enable themes and modules vary depending on the version of Drush that you have installed. This guide uses Drush 10. -{{< /note >}} - -## Installing Themes with Drush - -In this section you will download, enable, and set a Drupal theme using Drush. - -1. Navigate to [Drupal's Download and Extend page](https://www.drupal.org/project/project_theme) to find the theme you want to download. The Drush download name is usually appended to the end of the Drupal theme page's URL; for example, `drupal.org/project/project/my_theme`. - -1. Move into your Drupal site's document root, `/var/www/html/example.com/public_html/`. Replace `example.com` with your own domain's name. - - cd /var/www/html/example.com/public_html - -1. Download your desired theme using [Composer](https://getcomposer.org/doc/). For example to download the `bootstrap` theme use: - - composer require drupal/bootstrap - - {{< note respectIndent=false >}} -If you receive an error related to not being able to write to the `composer.json` file, see the [Setting the Site’s Ownership and Permissions](/cloud/guides/how-to-install-drupal-using-drush-on-ubuntu-18-04#setting-the-sites-ownership-and-permissions) section of the [Install Drupal using Drush on Ubuntu 18.04](/cloud/guides/how-to-install-drupal-using-drush-on-ubuntu-18-04) guide. - -Ensure that your `/var/www/html/example.com/public_html` directory has user and group read, write, and execute permissions. - - sudo chmod 774 -R /var/www/html/example.com/public_html - {{< /note >}} - -1. Enable the theme that you downloaded in the previous step. For example, to enable the `bootstrap` theme run the following command. Replace `bootstrap` with the name of your theme. - - drush theme:enable bootstrap - -1. As an example, set the [Omega](https://www.drupal.org/project/omega) theme as the default and active theme: - - drush config-set system.theme default omega - - Check the homepage of your site and the new theme should appear. - -## Installing Modules with Drush - -Downloading and enabling a module is similar to working with a theme. However, modules can be used for almost any purpose. From enhancing public-facing functionality to providing a better administrative UI; there are thousands of Drupal modules. It is helpful to use modules with clear documentation, since once a module is installed, its interface can be challenging to use if it is not well documented and designed by the contributor. - -1. Move into your Drupal site's document root, `/var/www/html/example.com/public_html/`. Replace `example.com` with your own domain's name. - - cd /var/www/html/example.com/public_html - -1. Download the [Backup and Migrate](https://www.drupal.org/project/backup_migrate) module. This module enables you to perform the backup, restore, and migrate tasks - - composer require drupal/backup_migrate - -1. Enable the `backup_migrate` module: - - drush en backup_migrate -y - -1. Sign in to your Drupal site's browser interface and navigate to the **Extend** menu item (or **Administration > Extend**). Under the **OTHER** section, the Backup and Migrate module will be listed and selected. - -You have successfully installed and enabled a new module. The module is now running and ready to be used. - -## Backup a Drupal Site - -It's always important to keep regular backups of a website. Backups protect you from losing data due to configuration changes, vulnerabilities, or system failures. Backups should be stored on a separate system whenever possible. The **Backup and Migrate** module helps you to create backups of your site. - -1. To configure backup and migrate, navigate to **Administration > Configuration > Development > Backup and Migrate** - -1. For a quick backup, select the type of **Backup Source** and select the **Backup Destination**, and click **Backup now** -{{< note >}} -Always download a backup prior to updating or installing modules. -{{< /note >}} diff --git a/docs/guides/websites/cms/drupal/how-to-install-drupal-with-docker-compose-ubuntu-18-04/index.md b/docs/guides/websites/cms/drupal/how-to-install-drupal-with-docker-compose-ubuntu-18-04/index.md deleted file mode 100644 index 39968c46956..00000000000 --- a/docs/guides/websites/cms/drupal/how-to-install-drupal-with-docker-compose-ubuntu-18-04/index.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -slug: how-to-install-drupal-with-docker-compose-ubuntu-18-04 -title: Installing Drupal with Docker Compose on Ubuntu 18.04 -title_meta: How to Install Drupal with Docker Compose on Ubuntu 18.04 -description: 'This guide will show you how to install Drupal 8, a popular free and open-source content management system, with Docker Compose on Ubuntu 18.04.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-02-18 -keywords: ["cms", "apache", "php", "content management system", "drupal 8", "ubuntu 18.04", "docker compose"] -tags: ["drupal","docker","container","lamp","cms","debian"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Drupal_DockerComp_Ubuntu1804.png -relations: - platform: - key: how-to-install-drupal-docker-compose - keywords: - - distribution: Ubuntu 18.04 -aliases: [] ---- - -## What Are Docker and Docker Compose? - -**Docker** is a system that provides pre-configured, self-contained applications, frameworks, and software stacks, such as WordPress, Golang, or LAMP. Even entire Linux distributions can be run in Docker. When deployed, these software packages are referred to as *containers*. Docker also allows you to create your own containers that include any custom software you'd like. - -**Docker Compose** is a complementary system which helps you link together individual Docker containers so they can work together. This guide walks through the deployment of a Drupal container and another PostgreSQL container that Drupal will use to store its data. Docker Compose will facilitate the networking between them. - -Containers for [Drupal](https://www.drupal.org/) and [PostgreSQL](https://www.postgresql.org/) are available from [Docker Hub](https://hub.docker.com/) in the form of *images*. A Docker image is a static snapshot of a container which is used to create new container instances. Docker Hub is an official repository where individuals and organizations can upload Docker images for public consumption. - -## Why Use Docker to Run Drupal? - -Using the Drupal and PostgreSQL images from Docker Hub offers the following benefits: - -- The configuration of the software has been done for you, which means that you don't need to follow a step-by-step process for each application to get them running on your system. -- Updating your software is as simple as downloading the latest images from Docker Hub. -- Images and containers are self-contained, which means that they are easy to clean up if you decide to remove them. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for [setting your Linode's hostname](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-custom-hostname) and [timezone](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#set-the-timezone). - -1. Follow our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to [create a standard user account](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#add-a-limited-user-account), [harden SSH access](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#harden-ssh-access), and [create firewall rules](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance#configure-a-firewall) for your web server; you may need to make additional firewall exceptions for your specific application. - - {{% content "limited-user-note-shortguide" %}} - -### Install Docker - -{{% content "installing-docker-shortguide" %}} - -### Install Docker Compose - -{{% content "install-docker-compose" %}} - -## Set Up Drupal - -1. Create a new directory in your home folder called `my_drupal` and `cd` into it: - - mkdir ~/my_drupal/ - cd ~/my_drupal/ - -1. Create a file named `docker-compose.yml` in this folder and add the following contents. Set your own password for the `POSTGRES_PASSWORD` option. - - {{< file "docker-compose.yml" yaml >}} -version: '3.3' - -services: - drupal: - image: drupal:latest - ports: - - 80:80 - volumes: - - drupal_modules:/var/www/html/modules - - drupal_profiles:/var/www/html/profiles - - drupal_themes:/var/www/html/themes - - drupal_sites:/var/www/html/sites - restart: always - - postgres: - image: postgres:10 - environment: - POSTGRES_PASSWORD: your_postgres_password - volumes: - - db_data:/var/lib/postgresql/data - restart: always - -volumes: - drupal_modules: - drupal_profiles: - drupal_themes: - drupal_sites: - db_data: -{{< /file >}} - -1. From the `my_drupal` directory, start your Docker containers: - - docker-compose up -d - -1. The Docker containers will take a minute or two to start up Drupal and PostgreSQL. Afterwards, you can visit your Linode's IP address in your web browser and you should be directed to the Drupal setup form. - - ![Drupal setup screen in the web browser - 'Choose language' page](drupal_setup_choose_language.png "Drupal setup screen in the web browser - 'Choose language' page") - -1. On the **Set up database** page, select `PostgreSQL` as the *Database type* and enter the following values: - - - Database name: `postgres` - - - Database username: `postgres` - - - Database password: *The password you set in the docker-compose.yml file* - - - Host (under Advanced Options): `postgres` - - ![Drupal setup screen in the web browser - 'Set up database' page](drupal_setup_set_up_database.png "Drupal setup screen in the web browser - 'Set up database' page") - -1. Complete the other screens in the setup guide. When creating your Drupal user, be sure to enter a password that is different from your PostgreSQL password. - -## Usage and Maintenance - -You do not need to manually start your containers if you reboot your Linode, because the option `restart: always` was assigned to your services in your `docker-compose.yml` file. This option tells Docker Compose to automatically start your services when the server boots. - -### Stop Drupal - -To stop your Drupal application: - - cd ~/my_drupal/ - docker-compose stop - -This will stop the running Drupal and PostgreSQL containers, but will not remove them. - -### Restart Drupal - -To restart your Drupal application: - - cd ~/my_drupal/ - docker-compose start - -### Stop and Remove Drupal - -To stop and remove containers, networks and images created by the `docker-compose.yml` file: - - cd ~/my_drupal/ - docker-compose down - -When a Docker container is taken down, it is also deleted; this is how Docker is designed to work. However, your Drupal files and data will be preserved, as the `docker-compose.yml` file was configured to create persistent volumes for that data. - -If you want to remove this data and start over with your Drupal site, you can add the `--volumes` flag to the previous command. **This will permanently delete the Drupal customizations you've made so far.** - - docker-compose down --volumes - -### Update Drupal - -The `docker-compose.yml` specifies the `latest` version of the Drupal image, so it's easy to update your Drupal version: - - docker-compose down - docker-compose pull && docker-compose up -d - -## Next Steps - -More extensive documentation on Docker is available in the [Containers](/cloud/guides/applications/containers) section of the Linode Guides & Tutorials site. diff --git a/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-centos-8/index.md b/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-centos-8/index.md deleted file mode 100644 index 1f3315c0045..00000000000 --- a/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-centos-8/index.md +++ /dev/null @@ -1,146 +0,0 @@ ---- -slug: how-to-update-drupal-8-on-centos-8 -title: Update Drupal 8 on CentOS 8 -title_meta: How to Update Drupal 8 on CentOS 8 -description: 'This guide will show you how to update your Drupal 8 installation running on a CentOS 8 Linode.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-02-21 -keywords: ["cms", "apache", "php", "content management system", "drupal 8", "update"] -tags: ["drupal","centos","cms","lamp"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Drupal8onCentOS8.png -relations: - platform: - key: how-to-update-drupal-8 - keywords: - - distribution: CentOS 8 -aliases: [] ---- - -Drupal 8 is the latest version of the popular [Drupal](https://www.drupal.org/) content management system. While Drupal 8.1 includes a simple feature for incremental updates, you must preform manual Drupal core updates for any preceding versions. This guide demonstrates how to manually install an incremental Drupal 8 update on your Linode. The examples in this guide assume you have a functional [Drupal 8 installation](/cloud/guides/how-to-install-and-configure-drupal-on-centos-8) running a [LAMP stack](/cloud/guides/how-to-install-a-lamp-stack-on-centos-8) on CentOS 8. - -## Before You Begin - -1. Complete all the steps in the [Install and Configure Drupal 8 on CentOS 8](/cloud/guides/how-to-install-and-configure-drupal-on-centos-8) guide. - -1. If you followed the [Install and Configure Drupal 8 on CentOS 8](/cloud/guides/how-to-install-and-configure-drupal-on-centos-8) guide, your site's document root should be in the `/var/www/html/example.com/` directory, where `example.com` is your own site's domain name. You can list all your directories in `/var/www/html` to verify the location of your site's document root. - - ls /var/wwww/html/ - -1. Update your CentOS 8 system if you did not complete that step while installing Drupal 8. - - sudo yum update - - {{% content "limited-user-note-shortguide" %}} - -## Create Backups - -In this section, you will create an archive of your Drupal site's files and store the archive in a `backups` directory. If needed, you could extract the compressed files in your backup archive to restore the state of your site prior to updating it. - -1. Create a `backups` directory in your site's document root to store any backups you make of your Drupal site. - - sudo mkdir /var/www/html/example.com/backups - -1. Create an archive of your existing site files and move it into the `backups` directory. Ensure you replace `example.com` with your own site's domain name. - - cd /var/www/html/example.com/public_html - sudo tar -cvzf example.com-BCKP-$(date +%Y%m%d).tar.gz ./ - sudo mv -v example.com-BCKP-*.tar.gz ../backups - - {{< note respectIndent=false >}} -This process can also be scripted and run on a regular basis using [cron](/cloud/guides/schedule-tasks-with-cron). - {{< /note >}} - -## Download Updates - -You are now ready to check your Drupal system for available updates. Once you have identified the necessary updates, you will download them as an archive to your Linode. - -1. Log in to your Drupal site and navigate to the [Admin Toolbar](https://www.drupal.org/project/admin_toolbar). Click on **Reports** and then on **Available updates**. - - {{< note respectIndent=false >}} -If **Available updates** is not listed, enable the [Update Manager](https://www.drupal.org/docs/8/core/modules/update-manager) module by navigating to the **Extend** menu item in the Admin Toolbar. See [Drupal's documentation](https://www.drupal.org/docs/8/extending-drupal-8/installing-drupal-8-modules#s-step-2-enable-the-module) for more details on enabling modules. -{{< /note >}} - -1. Right click the link under the **RECOMMENDED VERSION** heading and copy the link address and paste it somewhere you can access later. - - ![A Drupal Update](drupal-updates-download.png) - - {{< note respectIndent=false >}} -If you receive an error when your Drupal 8 installation checks for available updates, it may be having issues communicating with the Drupal website to see if there are updates. You can check your site's recent log messages, by navigating to **Reports** and selecting **Recent log messages** to further investigate the issue. - -If your CentOS installation is running in enforcing mode, ensure you are allowing httpd to make network connections. One way to do this is to set the corresponding SELinux boolean to `true`: - - sudo setsebool httpd_can_network_connect true - -Also, ensure that firewalld is allowing `https` traffic: - - sudo firewall-cmd --zone=public --add-service=https - {{< /note >}} - -1. Connect to your Linode over SSH: - - ssh user@192.0.2.0 - -1. Navigate to your site's directory. Download the Drupal core archive using `wget` and pasting the link address you copied from Step 2. Ensure you replace `example.com` with your own site's directory name. - - cd /var/www/html/example.com - sudo wget https://ftp.drupal.org/files/projects/drupal-8.8.2.tar.gz - -## Upgrade Your Site - -Now that the Drupal core archive is saved to your Linode, you are ready to proceed with the version upgrade of your site. - -### Enable Maintenance Mode - -Drupal's *maintenance mode* allows users with the right permissions to use your site while everyone else will be presented with a message that the site is under maintenance. - -1. While logged into your Drupal site, navigate to **Configuration**. Under the **Development** heading, click on **Maintenance mode**. - - ![Maintenance Mode](drupal-updates-maintenance.png) - -1. Check the box next to "Put site into maintenance mode." Enter a message if desired, and click **Save Configuration**. - - ![Turn on Maintenance Mode](drupal-updates-maintenance2.png) - -### Replace System Files - -1. Toggle back to your terminal window and ensure you are connected to your Linode over SSH. - - ssh user@192.0.2.0 - -1. Navigate to your site's `/var/www/html/example.com/public_html` directory and remove existing files and folders **except** `sites` and `profiles`. Ensure you replace `example.com` with your own site's directory name. - - cd /var/www/html/example.com/public_html - sudo rm -ifr autoload.php composer.* example.gitignore index.php LICENSE.txt README.txt robots.txt update.php web.config && sudo rm -ifr core/ modules/ vendor/ themes/ - -1. Go up one directory to `/var/www/html/example.com/` and expand the Drupal core archive that you downloaded in the [Download Updates](#download-updates) section into your `public_html` folder. Replace `drupal-8.8.2.tar.gz` with the name of the archive you downloaded: - - cd .. - sudo tar -zxvf drupal-8.8.2.tar.gz --strip-components=1 -C public_html - -1. From a browser on your local machine, navigate to the following URL on your Drupal site `www.example.com/update.php`. Ensure you replace `example.com` with your own site's domain name. Follow the prompts to continue the update. - - {{< note respectIndent=false >}} -If `update.php` does not load or returns a 403 Forbidden error, you may need to update the ownership and permissions of the newly expanded files. For best practices on Drupal site directory and file permissions, see their [documentation](https://www.drupal.org/node/244924). -{{< /note >}} - -1. If you are [installing additional modules](https://www.drupal.org/docs/user_guide/en/extend-module-install.html) or configuring additional [security settings](https://www.drupal.org/security/secure-configuration), complete those updates now and continue on to the next step in this section when you are done. - - {{< note respectIndent=false >}} -The [Next Steps](#next-steps) section includes a list of Drupal security modules you may consider installing. - {{< /note >}} - -1. Rebuild the site's cache by navigating to the Admin Toolbar and click on **Configuration**. Under the **Development** heading, click on **Performance**. Finally, click on the **Clear all caches** button. - -1. Verify your site's status by viewing its status report. In the Admin Toolbar, click on **Reports**, then click on **Status report**. - -1. If the update was successful and your status report does not display any unexpected information, take the site out of maintenance mode by following step 2 in the [Enable Maintenance Mode](#enable-maintenance-mode) section. Ensure you uncheck the box next to the "Put site into maintenance mode" setting. - -## Next Steps - -As a next step, consider installing additional security modules from the [Drupal Project Module](https://www.drupal.org/project/project_module): - -* [Secure Login](https://www.drupal.org/project/securelogin) enforces secure authenticated session cookies -* [Password Policy](https://www.drupal.org/project/password_policy) defines a user password policy -* [Security Review](https://www.drupal.org/project/security_review) automates security testing diff --git a/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-debian-10/index.md b/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-debian-10/index.md deleted file mode 100644 index 66df2d52175..00000000000 --- a/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-debian-10/index.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -slug: how-to-update-drupal-8-on-debian-10 -title: Update Drupal 8 on Debian 10 -title_meta: How to Update Drupal 8 on Debian 10 -description: 'This guide will show you how to update your Drupal 8 installation running on a Debian 10 Linode.' -og_description: 'This guide will show you how to update your Drupal 8 installation running on an Debian 10 Linode.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-02-21 -keywords: ["cms", "apache", "php", "content management system", "drupal 8", "update"] -tags: ["drupal","lamp","cms","debian"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Drupal8onDebian10.png -relations: - platform: - key: how-to-update-drupal-8 - keywords: - - distribution: Debian 10 -aliases: [] ---- - -Drupal 8 is the latest version of the popular [Drupal](https://www.drupal.org/) content management system. While Drupal 8.1 includes a simple feature for incremental updates, you must preform manual Drupal core updates for any preceding versions. This guide demonstrates how to manually install an incremental Drupal 8 update on your Linode. The examples in this guide assume you have a functional [Drupal 8 installation](/cloud/guides/how-to-install-and-configure-drupal-on-debian-10) running a [LAMP stack](/cloud/guides/how-to-install-a-lamp-stack-on-debian-10) on Debian 10. - -## Before You Begin - -1. Complete all the steps in the [Install and Configure Drupal 8 on Debian 10](/cloud/guides/how-to-install-and-configure-drupal-on-debian-10) guide. - -1. If you followed the [Install and Configure Drupal 8 on Debian 10](/cloud/guides/how-to-install-and-configure-drupal-on-debian-10) guide, your site's document root should be in the `/var/www/html/example.com/` directory, where `example.com` is your own site's domain name. You can list all your directories in `/var/www/html` to verify the location of your site's document root. - - ls /var/wwww/html/ - -1. Update your Debian 10 system if you did not complete that step while installing Drupal 8. - - sudo apt-get update && sudo apt-get upgrade - - {{% content "limited-user-note-shortguide" %}} - -## Create Backups - -In this section, you will create an archive of your Drupal site's files and store the archive in a `backups` directory. If needed, you could extract the compressed files in your backup archive to restore the state of your site prior to updating it. - -1. Create a `backups` directory in your site's document root to store any backups you make of your Drupal site. - - sudo mkdir /var/www/html/example.com/backups - -1. Create an archive of your existing site files and move it into the `backups` directory. Ensure you replace `example.com` with your own site's domain name. - - cd /var/www/html/example.com/public_html - sudo tar -cvzf example.com-BCKP-$(date +%Y%m%d).tar.gz ./ - sudo mv -v example.com-BCKP-*.tar.gz ../backups - - {{< note respectIndent=false >}} -This process can also be scripted and run on a regular basis using [cron](/cloud/guides/schedule-tasks-with-cron). - {{< /note >}} - -## Download Updates - -You are now ready to check your Drupal system for available updates. Once you have identified the necessary updates, you will download them as an archive to your Linode. - -1. Log in to your Drupal site and navigate to the [Admin Toolbar](https://www.drupal.org/project/admin_toolbar). Click on **Reports** and then on **Available updates**. - - {{< note respectIndent=false >}} -If **Available updates** is not listed, enable the [Update Manager](https://www.drupal.org/docs/8/core/modules/update-manager) module by navigating to the **Extend** menu item in the Admin Toolbar. See [Drupal's documentation](https://www.drupal.org/docs/8/extending-drupal-8/installing-drupal-8-modules#s-step-2-enable-the-module) for more details on enabling modules. -{{< /note >}} - -1. Right click the link under the **RECOMMENDED VERSION** heading and copy the link address and paste it somewhere you can access later. - - ![A Drupal Update](drupal-updates-download.png) - - {{< note respectIndent=false >}} -If you receive an error when your Drupal 8 installation checks for available updates, it may be having issues communicating with the Drupal website to see if there are updates. You can check your site's recent log messages, by navigating to **Reports** and selecting **Recent log messages** to further investigate the issue. - {{< /note >}} - -1. Connect to your Linode over SSH: - - ssh user@192.0.2.0 - -1. Navigate to your site's directory. Download the Drupal core archive using `wget` and pasting the link address you copied from Step 2. Ensure you replace `example.com` with your own site's directory name. - - cd /var/www/html/example.com - sudo wget https://ftp.drupal.org/files/projects/drupal-8.8.2.tar.gz - -## Upgrade Your Site - -Now that the Drupal core archive is saved to your Linode, you are ready to proceed with the version upgrade of your site. - -### Enable Maintenance Mode - -Drupal's *maintenance mode* allows users with the right permissions to use your site while everyone else will be presented with a message that the site is under maintenance. - -1. While logged into your Drupal site, navigate to **Configuration**. Under the **Development** heading, click on **Maintenance mode**. - - ![Maintenance Mode](drupal-updates-maintenance.png) - -1. Check the box next to "Put site into maintenance mode." Enter a message if desired, and click **Save Configuration**. - - ![Turn on Maintenance Mode](drupal-updates-maintenance2.png) - -### Replace System Files - -1. Toggle back to your terminal window and ensure you are connected to your Linode over SSH. - - ssh user@192.0.2.0 - -1. Navigate to your site's `/var/www/html/example.com/public_html` directory and remove existing files and folders **except** `sites` and `profiles`. Ensure you replace `example.com` with your own site's directory name. - - cd /var/www/html/example.com/public_html - sudo rm -ifr autoload.php composer.* example.gitignore index.php LICENSE.txt README.txt robots.txt update.php web.config && sudo rm -ifr core/ modules/ vendor/ themes/ - -1. Go up one directory to `/var/www/html/example.com/` and expand the Drupal core archive that you downloaded in the [Download Updates](#download-updates) section into your `public_html` folder. Replace `drupal-8.8.2.tar.gz` with the name of the archive you downloaded: - - cd .. - sudo tar -zxvf drupal-8.8.2.tar.gz --strip-components=1 -C public_html - -1. From a browser on your local machine, navigate to the following URL on your Drupal site `www.example.com/update.php`. Ensure you replace `example.com` with your own site's domain name. Follow the prompts to continue the update. - - {{< note respectIndent=false >}} -If `update.php` does not load or returns a 403 Forbidden error, you may need to update the ownership and permissions of the newly expanded files. For best practices on Drupal site directory and file permissions, see their [documentation](https://www.drupal.org/node/244924). -{{< /note >}} - -1. If you are [installing additional modules](https://www.drupal.org/docs/user_guide/en/extend-module-install.html) or configuring additional [security settings](https://www.drupal.org/security/secure-configuration), complete those updates now and continue on to the next step in this section when you are done. - - {{< note respectIndent=false >}} -The [Next Steps](#next-steps) section includes a list of Drupal security modules you may consider installing. - {{< /note >}} - -1. Rebuild the site's cache by navigating to the Admin Toolbar and click on **Configuration**. Under the **Development** heading, click on **Performance**. Finally, click on the **Clear all caches** button. - -1. Verify your site's status by viewing its status report. In the Admin Toolbar, click on **Reports**, then click on **Status report**. - -1. If the update was successful and your status report does not display any unexpected information, take the site out of maintenance mode by following step 2 in the [Enable Maintenance Mode](#enable-maintenance-mode) section. Ensure you uncheck the box next to the "Put site into maintenance mode" setting. - -## Next Steps - -As a next step, consider installing additional security modules from the [Drupal Project Module](https://www.drupal.org/project/project_module): - -* [Secure Login](https://www.drupal.org/project/securelogin) enforces secure authenticated session cookies -* [Password Policy](https://www.drupal.org/project/password_policy) defines a user password policy -* [Security Review](https://www.drupal.org/project/security_review) automates security testing diff --git a/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-ubuntu-18-04/index.md b/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-ubuntu-18-04/index.md deleted file mode 100644 index 5ccc931e063..00000000000 --- a/docs/guides/websites/cms/drupal/how-to-update-drupal-8-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -slug: how-to-update-drupal-8-on-ubuntu-18-04 -title: Update Drupal 8 on Ubuntu 18.04 -title_meta: How to Update Drupal 8 on Ubuntu 18.04 -description: 'This guide will show you how to update your Drupal 8 installation running on an Ubuntu 18.04 Linode.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-02-21 -keywords: ["cms", "apache", "php", "content management system", "drupal 8", "update"] -tags: ["drupal","ubuntu","cms","lamp"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: Drupal8onUbuntu1804.png -relations: - platform: - key: how-to-update-drupal-8 - keywords: - - distribution: Ubuntu 18.04 -aliases: [] ---- - -Drupal 8 is the latest version of the popular [Drupal](https://www.drupal.org/) content management system. While Drupal 8.1 includes a simple feature for incremental updates, you must preform manual Drupal core updates for any preceding versions. This guide demonstrates how to manually install an incremental Drupal 8 update on your Linode. The examples in this guide assume you have a functional [Drupal 8 installation](/cloud/guides/how-to-install-and-configure-drupal-on-ubuntu-18-04) running a [LAMP stack](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-18-04) on Ubuntu 18.04. - -{{< note >}} -If you are not using Ubuntu 18.04, you can find a version of this guide for your Linux distribution in the [Drupal](/cloud/guides/websites/cms/drupal) section of our documentation site. -{{< /note >}} - -## Before You Begin - -1. Complete all the steps in the [Install and Configure Drupal 8 on Ubuntu 18.04](/cloud/guides/how-to-install-and-configure-drupal-on-ubuntu-18-04) guide. - -1. If you followed the [Install and Configure Drupal 8 on Ubuntu 18.04](/cloud/guides/how-to-install-and-configure-drupal-on-ubuntu-18-04) guide, your site's document root should be in the `/var/www/html/example.com/` directory, where `example.com` is your own site's domain name. You can list all your directories in `/var/www/html` to verify the location of your site's document root. - - ls /var/wwww/html/ - -1. Update your Ubuntu 18.04 system if you did not complete that step while installing Drupal 8. - - sudo apt-get update && sudo apt-get upgrade - - {{% content "limited-user-note-shortguide" %}} - -## Create Backups - -In this section, you will create an archive of your Drupal site's files and store the archive in a `backups` directory. If needed, you could extract the compressed files in your backup archive to restore the state of your site prior to updating it. - -1. Create a `backups` directory in your site's document root to store any backups you make of your Drupal site. - - sudo mkdir /var/www/html/example.com/backups - -1. Create an archive of your existing site files and move it into the `backups` directory. Ensure you replace `example.com` with your own site's domain name. - - cd /var/www/html/example.com/public_html - sudo tar -cvzf example.com-BCKP-$(date +%Y%m%d).tar.gz ./ - sudo mv -v example.com-BCKP-*.tar.gz ../backups - - {{< note respectIndent=false >}} -This process can also be scripted and run on a regular basis using [cron](/cloud/guides/schedule-tasks-with-cron). - {{< /note >}} - -## Download Updates - -You are now ready to check your Drupal system for available updates. Once you have identified the necessary updates, you will download them as an archive to your Linode. - -1. Log in to your Drupal site and navigate to the [Admin Toolbar](https://www.drupal.org/project/admin_toolbar). Click on **Reports** and then on **Available updates**. - - {{< note respectIndent=false >}} -If **Available updates** is not listed, enable the [Update Manager](https://www.drupal.org/docs/8/core/modules/update-manager) module by navigating to the **Extend** menu item in the Admin Toolbar. See [Drupal's documentation](https://www.drupal.org/docs/8/extending-drupal-8/installing-drupal-8-modules#s-step-2-enable-the-module) for more details on enabling modules. -{{< /note >}} - -1. Right click the link under the **RECOMMENDED VERSION** heading and copy the link address and paste it somewhere you can access later. - - ![A Drupal Update](drupal-updates-download.png) - - {{< note respectIndent=false >}} -If you receive an error when your Drupal 8 installation checks for available updates, it may be having issues communicating with the Drupal website to see if there are updates. You can check your site's recent log messages, by navigating to **Reports** and selecting **Recent log messages** to further investigate the issue. - {{< /note >}} - -1. Connect to your Linode over SSH: - - ssh user@192.0.2.0 - -1. Navigate to your site's directory. Download the Drupal core archive using `wget` and pasting the link address you copied from Step 2. Ensure you replace `example.com` with your own site's directory name. - - cd /var/www/html/example.com - sudo wget https://ftp.drupal.org/files/projects/drupal-8.8.2.tar.gz - -## Upgrade Your Site - -Now that the Drupal core archive is saved to your Linode, you are ready to proceed with the version upgrade of your site. - -### Enable Maintenance Mode - -Drupal's *maintenance mode* allows users with the right permissions to use your site while everyone else will be presented with a message that the site is under maintenance. - -1. While logged into your Drupal site, navigate to **Configuration**. Under the **Development** heading, click on **Maintenance mode**. - - ![Maintenance Mode](drupal-updates-maintenance.png) - -1. Check the box next to "Put site into maintenance mode." Enter a message if desired, and click **Save Configuration**. - - ![Turn on Maintenance Mode](drupal-updates-maintenance2.png) - -### Replace System Files - -1. Toggle back to your terminal window and ensure you are connected to your Linode over SSH. - - ssh user@192.0.2.0 - -1. Navigate to your site's `/var/www/html/example.com/public_html` directory and remove existing files and folders **except** `sites` and `profiles`. Ensure you replace `example.com` with your own site's directory name. - - cd /var/www/html/example.com/public_html - sudo rm -ifr autoload.php composer.* example.gitignore index.php LICENSE.txt README.txt robots.txt update.php web.config && sudo rm -ifr core/ modules/ vendor/ themes/ - -1. Go up one directory to `/var/www/html/example.com/` and expand the Drupal core archive that you downloaded in the [Download Updates](#download-updates) section into your `public_html` folder. Replace `drupal-8.8.2.tar.gz` with the name of the archive you downloaded: - - cd .. - sudo tar -zxvf drupal-8.8.2.tar.gz --strip-components=1 -C public_html - -1. From a browser on your local machine, navigate to the following URL on your Drupal site `www.example.com/update.php`. Ensure you replace `example.com` with your own site's domain name. Follow the prompts to continue the update. - - {{< note respectIndent=false >}} -If `update.php` does not load or returns a 403 Forbidden error, you may need to update the ownership and permissions of the newly expanded files. For best practices on Drupal site directory and file permissions, see their [documentation](https://www.drupal.org/node/244924). -{{< /note >}} - -1. If you are [installing additional modules](https://www.drupal.org/docs/user_guide/en/extend-module-install.html) or configuring additional [security settings](https://www.drupal.org/security/secure-configuration), complete those updates now and continue on to the next step in this section when you are done. - - {{< note respectIndent=false >}} -The [Next Steps](#next-steps) section includes a list of Drupal security modules you may consider installing. - {{< /note >}} - -1. Rebuild the site's cache by navigating to the Admin Toolbar and click on **Configuration**. Under the **Development** heading, click on **Performance**. Finally, click on the **Clear all caches** button. - -1. Verify your site's status by viewing its status report. In the Admin Toolbar, click on **Reports**, then click on **Status report**. - -1. If the update was successful and your status report does not display any unexpected information, take the site out of maintenance mode by following step 2 in the [Enable Maintenance Mode](#enable-maintenance-mode) section. Ensure you uncheck the box next to the "Put site into maintenance mode" setting. - -## Next Steps - -As a next step, consider installing additional security modules from the [Drupal Project Module](https://www.drupal.org/project/project_module): - -* [Secure Login](https://www.drupal.org/project/securelogin) enforces secure authenticated session cookies -* [Password Policy](https://www.drupal.org/project/password_policy) defines a user password policy -* [Security Review](https://www.drupal.org/project/security_review) automates security testing diff --git a/docs/guides/websites/cms/ghost/how-to-install-ghost-on-centos-8/index.md b/docs/guides/websites/cms/ghost/how-to-install-ghost-on-centos-8/index.md deleted file mode 100644 index be56c4b05a6..00000000000 --- a/docs/guides/websites/cms/ghost/how-to-install-ghost-on-centos-8/index.md +++ /dev/null @@ -1,285 +0,0 @@ ---- -slug: how-to-install-ghost-on-centos-8 -title: Installing Ghost CMS on CentOS 8 -title_meta: How to Install Ghost CMS on CentOS 8 -description: 'This tutorial will teach you how to install Ghost, a publishing platform great for running blogs and sharing published content, on CentOS 8.' -og_description: 'Easily publish your own professional-looking blog using Ghost on your Linode running CentOS 8.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2020-02-10 -keywords: ["ghost", "install ghost", "ghost on linode", "configure ghost", "deploy ghost on CentOS 8", "ghost cms"] -tags: ["nginx","centos","mysql","cms"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: GhostCMSonCentos8.png -external_resources: -- '[Ghost Setup Documentation](https://ghost.org/docs/setup/)' -- '[Ghost Theme Documentation](https://ghost.org/docs/api/v3/handlebars-themes/)' -- '[Ghost API Documentation](https://ghost.org/docs/api/v3/)' -relations: - platform: - key: how-to-install-ghost-cms - keywords: - - distribution: CentOS 8 -aliases: [] ---- - -[Ghost](https://ghost.org/developers/) is an open source blogging platform that helps you easily create a professional-looking online blog. Ghost is a robust content management system (CMS) with a Markdown editor, an easy-to-use user interface, and beautiful themes. It is easy to install and update with [Ghost-CLI](https://github.com/TryGhost/Ghost-CLI). - -## In This Guide - -In this guide, you'll set up, deploy, and secure a Ghost v3.5.1 blog on a Linode running CentOS 8, using NGINX, MariaDB, Node.js, NPM, and Ghost-CLI. For installation instructions for other distributions, click [here](/cloud/guides/websites/cms/ghost). - -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, consult our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. - -Replace each instance of `example.com` in this guide with your site’s domain name. -{{< /note >}} - -## Before you Begin - -1. This guide assumes that you've followed the steps in our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guides and have created a new user for Ghost with elevated `sudo` privileges. The example username used in this guide is `ghostexample`. - -1. Ensure that you have a valid domain name and [properly configured DNS records](https://techdocs.akamai.com/cloud-computing/docs/dns-manager) for your domain. - -1. Ensure that your system is up to date: - - sudo yum update - -1. Set your system to SELinux permissive mode: - - sudo setenforce 0 - sudo sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config - -1. Add `EPEL repository`: - - sudo dnf -y install epel-release - -## Install Prerequisites - -### Install and Configure MariaDB - -1. Download and install MariaDB: - - sudo dnf module install mariadb - -1. Set MySQL to start on reboot: - - sudo systemctl enable --now mariadb - -1. Secure the MySQL installation: - - mysql_secure_installation - -You will be given the choice to change the MariaDB root password, remove anonymous user accounts, disable root logins outside of localhost, and remove test databases. It is recommended that you answer `yes` to these options. You can read more about the script in the [MariaDB Knowledge Base](https://mariadb.com/kb/en/mariadb/mysql_secure_installation/). - -### Install Node.js and NPM - -Ghost is built on Node.js and follows Node's Long Term Support (LTS) plan. Ghost only supports [LTS versions of Node.js](https://github.com/nodejs/LTS). - -Download and install Node.js: - - sudo dnf -y install @nodejs - sudo npm install pm2 -g - -### Install and Configure NGINX - -NGINX will be used as a reverse proxy for your Ghost application: - -1. Install NGINX: - - sudo dnf install @nginx - -1. Navigate to the `/etc/nginx/` directory which will contain your configuration files: - - cd /etc/nginx/ - -1. Create new directories for your Ghost configuration files: - - sudo mkdir sites-available - sudo mkdir sites-enabled - -1. Create a configuration file for Ghost at `/etc/nginx/sites-available/ghost`, replace `example.com` with your site's domain: - - {{< file "/etc/nginx/sites-available/ghost" >}} -server { - listen 80 default_server; - listen [::]:80 default_server; - - server_name example.com www.example.com; - root /var/www/ghost; - - location / { - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header Host $host; - proxy_pass http://127.0.0.1:2368; - } - - client_max_body_size 50m; -} -{{}} - -1. Create a symlink from `sites-available` to `sites-enabled`: - - sudo ln -s /etc/nginx/sites-available/ghost /etc/nginx/sites-enabled/ghost - -1. Update the `http` block in your NGINX config `/etc/nginx/nginx.conf` file by adding an include for the configuration file you just made, a directive for `server_names_hash_bucket_size`, and comment out the entire `server` block: - - {{< file "/etc/nginx/nginx.conf" >}} -http { -... - # Load modular configuration files from the /etc/nginx/conf.d directory. - # See http://nginx.org/en/docs/ngx_core_module.html#include - # for more information. - include /etc/nginx/conf.d/*.conf; - include /etc/nginx/sites-enabled/*; - - server_names_hash_bucket_size 64; - -# server { -# listen 80 default_server { -# ... -# } -#} -... -{{}} - -1. Open the firewall for traffic: - - sudo firewall-cmd --zone=public --permanent --add-service=http - sudo firewall-cmd --zone=public --permanent --add-service=https - sudo firewall-cmd --reload - -1. You can test your NGINX configuration with this command: - - sudo nginx -t - -1. Start the service with the following commands: - - sudo systemctl enable nginx - sudo systemctl start nginx - -1. Verify that it's running: - - sudo systemctl status nginx - -## Install and Configure Ghost - -### Install Ghost-CLI - -Ghost-CLI is a command line interface (CLI) tool that makes installing and updating Ghost easy. For CentOS, it will set up the database and initialize Ghost as a systemd service. - -Install Ghost-CLI: - - sudo npm install -g ghost-cli@latest - -### Install Ghost - -Install Ghost using the Ghost-CLI tool. - -1. Create the document root directory: - - sudo mkdir -p /var/www/ghost - - {{< note respectIndent=false >}} -Installing Ghost in the `/root` or `/home/{user}` folder won’t work and results in a broken setup. Only use `/var/www/{folder}` because it has the correct permissions. -{{< /note >}} - -1. Change ownership of the `/var/www/ghost` directory to the non-root user with `sudo` privileges that you created. In this example, `ghostexample` is our username: - - sudo chown ghostexample:ghostexample /var/www/ghost - sudo chmod 775 /var/www/ghost - -1. Navigate to the Ghost root directory: - - cd /var/www/ghost - -1. Ensure that the directory is empty to avoid file conflicts: - - ls -a - -1. Install Ghost in production mode: - - ghost install - - {{< note respectIndent=false >}} -By default Ghost looks for Ubuntu and will display this warning if it detects a different operating system. - -{{< output >}} -System checks failed with message: 'Linux version is not Ubuntu 16 or 18' -Some features of Ghost-CLI may not work without additional configuration. -For local installs we recommend using `ghost install local` instead. -{{}} - -It will ask you if you would like to continue anyway, answer yes. - -The installer also gets a little confused at our installing MariaDB instead of MySQL. It appears to error with a note that it is skipping MySQL and asks if you want to continue anyway, answer yes. -{{< /note >}} - -1. Answer each question as prompted. For more information about each question, visit the [Ghost documentation](https://ghost.org/docs/install/ubuntu/#install-questions): - - {{< output >}} -? Enter your blog URL: https://example.com -? Enter your MySQL hostname: localhost -? Enter your MySQL username: root -? Enter your MySQL password: thePasswordYouEnteredForRoot -? Enter your Ghost database name: exampleGhost -Configuring Ghost -Setting up instance -+ sudo useradd --system --user-group ghost -+ sudo chown -R ghost:ghost /var/www/ghost/content -Setting up "ghost" system user -? Do you wish to set up "ghost" mysql user? yes -Nginx is not installed. Skipping Nginx setup. -i Setting up Nginx [ skipped ] -Nginx setup task was skipped, skipping SSL setup -i Setting up SSL [ skipped ] -? Do you wish to set up Systemd? yes -Creating systemd service file at /var/www/ghost/system/files/ghost_example-com.service -+ sudo ln -sf /var/www/ghost/system/files/ghost_example-com.service /lib/systemd/system/ghost_example-com.service -+ sudo systemctl daemon-reload -Setting up Systemd -+ sudo systemctl is-active ghost_example-com -? Do you want to start Ghost? yes -+ sudo systemctl start ghost_example-com -+ sudo systemctl is-enabled ghost_example-com -+ sudo systemctl enable ghost_example-com --quiet -Starting Ghost -{{< /output >}} - -1. After installation is complete, run `ghost ls` to view running Ghost processes: - - ghost ls - -In the future when a newer version of Ghost is released, run `ghost update` from the `/var/www/ghost` directory to update to the newest version. - -### Complete the Setup - -To complete the setup process, navigate to the Ghost configuration page by appending `/ghost` to the end of your blog’s URL or IP. This example uses `https://example.com/ghost`. - -1. On the welcome screen, click **Create your account**: - - ![Ghost Welcome Screen](ghost-welcome-screen.png "Ghost Welcome Screen") - -1. Enter your email, create a user, password, and blog title: - - ![Create Your Account Screen](ghost-create-your-account.png "Create Your Account Screen") - -1. Invite additional members to your team. If you’d prefer to skip this step, click **I’ll do this later, take me to my blog!** at the bottom of the page. - - ![Invite Your Team Screen](ghost-1-0-0-invite-team-small.png "Invite Your Team Screen") - -1. Navigate the Ghost admin area to create your first post, change your site's theme, or configure additional settings: - - ![Ghost Admin Area](ghost-admin-area.png "Ghost Admin Area") - -## Troubleshooting - -1. Troubleshoot the system for any potential issues when installing or updating Ghost: - - ghost doctor - -1. Get help about Ghost: - - ghost --help diff --git a/docs/guides/websites/cms/wordpress/how-to-install-wordpress-using-wp-cli-on-centos-7/index.md b/docs/guides/websites/cms/wordpress/how-to-install-wordpress-using-wp-cli-on-centos-7/index.md deleted file mode 100644 index 99995c75b77..00000000000 --- a/docs/guides/websites/cms/wordpress/how-to-install-wordpress-using-wp-cli-on-centos-7/index.md +++ /dev/null @@ -1,522 +0,0 @@ ---- -slug: how-to-install-wordpress-using-wp-cli-on-centos-7 -title: 'Install WordPress Using WP-CLI on CentOS 7' -title_meta: 'How to Install WordPress Using WP-CLI on CentOS 7' -description: 'Install WordPress Using WP-CLI on CentOS 7' -authors: ["Linode"] -contributors: ["Linode"] -published: 2018-08-06 -modified: 2020-02-17 -keywords: ["install WP-CLI", "centos", "wordpress", "apache", "bash completion", "plugin", "WP-CLI", "themes"] -tags: ["centos","wordpress","cms","lamp"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: -- '[WP-CLI Handbook](https://make.wordpress.org/cli/handbook/)' -- '[WP-CLI Commands](https://developer.wordpress.org/cli/commands/)' -- '[WP-CLI Tools](https://make.wordpress.org/cli/handbook/tools/)' -relations: - platform: - key: how-to-install-wordpress-using-wp-cli - keywords: - - distribution: CentOS 7 -aliases: [] ---- - -WordPress is well-known for its rich content management feature set, ease of use, and quick installation time. The [WordPress command line interface (WP-CLI)](https://wp-cli.org/) provides useful commands and utilities to install, configure, and manage a WordPress site. This guide walks you through some common tasks you can complete using the WP-CLI. - -## In this Guide: - -This tutorial covers how to complete the following tasks: - -- [Install the WP-CLI on a Linode running CentOS 7](#install-wp-cli) -- [Install a WordPress instance using the WP-CLI](#install-wordpress) -- [Helpful and common WP-CLI commands you can use to manage your WordPress site](#common-commands) - -## Prerequisites - -Before moving ahead, make sure you have completed the following steps. - -1. If you have not already done so, create a Linode account and Compute Instance. See our [Getting Started with Linode](https://techdocs.akamai.com/cloud-computing/docs/getting-started) and [Creating a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/create-a-compute-instance) guides. - -1. Follow our [Setting Up and Securing a Compute Instance](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to update your system. You may also wish to set the timezone, configure your hostname, create a limited user account, and harden SSH access. - - {{< note respectIndent=false >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -1. If you'd like to use your own [Domain Name](/cloud/guides/dns-overview) to host your WordPress installation, ensure that your domain name is [pre-configured](https://techdocs.akamai.com/cloud-computing/docs/getting-started-with-dns-manager) to point to your Linode's IP address. - -1. Follow the [Install a LAMP Stack on CentOS 7](/cloud/guides/how-to-install-a-lamp-stack-on-centos-7) guide. **Skip the steps** in the [Configure Apache Virtual Hosts File](/cloud/guides/how-to-install-wordpress-using-wp-cli-on-centos-7#configure-apache-virtual-hosts-file), the [Create a MariaDB Database](/cloud/guides/how-to-install-a-lamp-stack-on-centos-7#create-a-mariadb-database), and the [Optional: Test and Troubleshoot the LAMP Stack](/cloud/guides/how-to-install-a-lamp-stack-on-centos-7#optional-test-and-troubleshoot-the-lamp-stack) section. Those steps will be covered later on in this guide. - - {{< note respectIndent=false >}} -When following the steps to [install PHP](#install-php) in the [Install a LAMP Stack on CentOS 7](/cloud/guides/how-to-install-a-lamp-stack-on-centos-7) guide, you will need to issue the command included below to install the required PHP packages. The command in the linked guide does not currently work with CentOS 7. - - sudo apt install php libapache2-mod-php php-mysql - {{< /note >}} - -## Install WP-CLI - -1. WP-CLI is available as a PHP Archive file (`.phar`). You can download it using either `wget` or `curl` commands: - - curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar - - **Or** - - wget https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar - -1. You need to make this `.phar` file executable and move it to `/usr/local/bin` so that it can be run directly: - - chmod +x wp-cli.phar - sudo mv wp-cli.phar /usr/local/bin/wp - -1. Check if it is installed properly: - - wp --info - - You should see a similar output to the one displayed below, which means you can proceed: - - {{< output >}} -OS: Linux 4.15.0-50-generic #54-Ubuntu SMP Mon May 6 18:46:08 UTC 2019 x86_64 -Shell: /bin/bash -PHP binary: /usr/bin/php7.2 -PHP version: 7.2.24-0ubuntu0.18.04.2 -php.ini used: /etc/php/7.2/cli/php.ini -WP-CLI root dir: phar://wp-cli.phar/vendor/wp-cli/wp-cli -WP-CLI vendor dir: phar://wp-cli.phar/vendor -WP_CLI phar path: /home/lsalazar -WP-CLI packages dir: -WP-CLI global config: -WP-CLI project config: -WP-CLI version: 2.4.0 - {{}} - - You can use the above three steps for upgrading WP-CLI as well. - -### Activate Bash Completion - -The bash completion feature of WP-CLI allows you to see all its available commands on the fly when pressing **Tab**. WP-CLI supports tab completion for [Bash](https://www.gnu.org/software/bash/) and [Zsh](https://en.wikipedia.org/wiki/Z_shell). - -1. Install the wget package: - - sudo yum install wget -1. Download the bash script in your home directory: - - cd ~ - wget https://github.com/wp-cli/wp-cli/raw/master/utils/wp-completion.bash - -1. Edit your shell's configuration file so that wp-completion is loaded by the shell every time you open a new shell session: - - **Bash** - - Open the `.bashrc` file and add the following line to the bottom of the file: - - {{< file "~/.bashrc" bash >}} -source /home/$USER/wp-completion.bash -{{< /file >}} - - - Run the following command to reload the bash profile: - - source ~/.bashrc - - **Zsh** - - - Open the `.zshrc`file and add the following line to the bottom of the file: - - {{< file "~/.zshrc" bash >}} -autoload bashcompinit -bashcompinit -source /home/$USER/wp-completion.bash -{{< /file >}} - - - Run the following command to reload the Zsh profile: - - source ~/.zshrc - -1. Shell completion is now enabled. To test it, type `wp theme ` (include the trailing space) **Without Hitting Enter** and press **Tab** twice. You will see the list of available commands with `wp theme` again on the prompt. - -## Basics of WP-CLI - -In this section, you will learn some basics of how WP-CLI works. This will help you when using the tool in the upcoming sections of the guide. - -- So far, we have seen WP-CLI accessed through the main command, `wp`. You can follow the main command with nested subcommands. For example, WP-CLI includes a command to download WordPress: - - {{< note respectIndent=false >}} -Do not issue the example `wp` command. You will install WordPress in the [Download and Configure WordPress](#download-and-configure-wordpress) section of the guide. - {{< /note >}} - - wp core download - - In the example, `wp` is the main command while `core` and `download` are its nested subcommands. Nesting subcommands can extend one or two levels. - -- WP-CLI also comes with a detailed help section, which displays all the commands you might need. To access help enter the following command: - - wp help - - The output should resemble: - - {{< output >}} -wp - -DESCRIPTION - -Manage WordPress through the command-line. - -SYNOPSIS - -wp - -SUBCOMMANDS - -cache Manage the object cache. -cap Manage user capabilities. -cli Get information about WP-CLI itself. -comment Manage comments. -core Download, install, update and otherwise manage WordPress proper. -cron Manage WP-Cron events and schedules. -db Perform basic database operations. -eval Execute arbitrary PHP code after loading WordPress. -eval-file Load and execute a PHP file after loading WordPress. -: - {{}} - - - `:` is a prompt that, with subcommands, can help you navigate through this help menu. - - Up and down arrow keys will let you scroll through the entire help command list. - - Typing `q` will exit the help menu. - - For additional details on how to further navigate through the complete help section, you can always type `h` at the above prompt. - - You can use the enabled bash completion to demonstrate WP-CLI's readily available command list. Simply type `wp` and press tab twice. You will see the list of available commands. Now, type `wp core` and press tab twice. You will see a list of commands that can be used with `core`. This double tabbing after a command can be repeated for any primary or subcommand. - -## Install WordPress - -In this section, you will complete the prerequisite configuration steps needed to install WordPress. Then, you will install WordPress using the WP-CLI. - -### Prepare the WordPress Database - -1. Log in to the MariaDB command line as the database's root user: - - sudo mysql -u root - - {{< note respectIndent=false >}} -If you set up a password for MySQL, you would log in with the `-p` flag as well: - - sudo mysql -u root -p -{{< /note >}} - -1. Create the WordPress database: - - {{< highlight sql >}} -CREATE DATABASE wordpress; -{{< /highlight >}} - -1. Create a database user and grant them privileges for the newly created `wordpress` database, replacing `wpuser` and `password` with the username and password you wish to use: - - {{< highlight sql >}} -CREATE USER 'wpuser' IDENTIFIED BY 'password'; -GRANT ALL PRIVILEGES ON wordpress.* TO 'wpuser'; -FLUSH PRIVILEGES; -{{< /highlight >}} - -1. Type `quit` to exit the MariaDB command line. - -1. Install the `policycoreutils-python` package to manage SELinux and open your MySQL port: - - sudo yum install policycoreutils-python - -1. Configure SELinux to enable MariaDB's use of port 3306: - - sudo semanage port -a -t mysqld_port_t -p tcp 3306 - - {{< note respectIndent=false >}} -Use **netstat** to confirm the port used by MariaDB with the command `netstat -tln`. The port in use will typically be `3306`. - {{< /note >}} - -1. Configure SELinux to allow your Apache web server to make database connections: - - sudo setsebool httpd_can_network_connect_db 1 - -### Upgrade to PHP Version 5.6 - -The latest version of WordPress (5.3.2 at the time of writing this guide) requires at least PHP version 5.6.20. This section includes steps to upgrade your CentOS 7 Linode's PHP version, if needed. - -1. Check the version of PHP that you are currently running. If you see a version lower than 5.6.20, then proceed with the next steps: - - php --version - -1. Install and enable [EPEL](https://fedoraproject.org/wiki/EPEL) and [Remi](https://rpms.remirepo.net/) repositories on your system: - - sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm - sudo yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm - -1. Install the `yum-utils` package which includes utilities which you'll need to complete your PHP version upgrade: - - sudo yum install yum-utils - -1. Activate the PHP 5.6 Remi repository using the `yum-config-manager` utility: - - sudo yum-config-manager --enable remi-php56 - -1. Install PHP version 5.6: - - sudo yum install php php-mcrypt php-cli php-gd php-curl php-mysql php-ldap php-zip php-fileinfo - - You should now have a version of PHP that will be compatible with the latest version of WordPress. - -### Download and Configure WordPress - -1. Create your WordPress site's document root. Replace `example.com` with your site's name: - - sudo mkdir -p /var/www/html/example.com/public_html - -1. Move to the Apache `example.com` directory: - - cd /var/www/html/example.com - -1. Change the ownership of the `public_html` directory. Apache comes with its own `apache` user and group. As a recommended practice, you should change the ownership of your installation directory to `apache`: - - sudo chown -R apache:apache /var/www/html/example.com/public_html - sudo chmod g+w /var/www/html/example.com/public_html - - -1. Add your limited user account to the `apache` group: - - sudo usermod -a -G apache user - - {{< note respectIndent=false >}} -You may need to log out of your SSH session and log back in for the user group change to take effect. To verify issue the following command. You should see the `apache` group returned as one of the groups. - - groups - -{{< output >}} -user wheel apache -{{}} - {{< /note >}} - -1. Change the user and group ownership of the `/var/www` directory. This will allow the `apache` user to write to the directory when caching downloaded files to the `/var/www/.wp-cli/cache` directory: - - sudo chown apache:apache /var/www - -1. Download the WordPress files. - - cd /var/www/html/example.com/public_html/ - wp core download - -1. Create a `wp-config.php` file. Replace `wpuser` and `password` with your WordPress database user and password. Also, replace `example_hostname` with your Linode's hostname. `dbprefix` is entirely optional and can be omitted unless you need to change their default values. - - wp core config --dbname='wordpress' --dbuser='wpuser' --dbpass='password' --dbhost='example_hostname' --dbprefix='wp_' - -1. Run the installation. Replace `adminuser` with the username you'd like to login to WordPress, and replace `password` with a unique password. Replace `example.com` with your domain, or replace it with your IP address if you haven't set up a domain yet. - - wp core install --url='http://example.com' --title='Blog Title' --admin_user='adminuser' --admin_password='password' --admin_email='email@domain.com' - -### Configure Apache Virtual Hosts File - -You will need to configure Apache so that you can access your WordPress site from a browser. - -1. Create directories for your WordPress site's Apache error and access logs: - - sudo mkdir /var/log/httpd/example.com/ - -1. Create directories for Apache's virtual hosts file: - - sudo mkdir /etc/httpd/sites-available /etc/httpd/sites-enabled - -1. Create your WordPress site's Apache error and access log files: - - sudo touch /var/log/httpd/example.com/access.log - sudo touch /var/log/httpd/example.com/error.log - -1. Edit Apache's configuration file to let it know to look for virtual host files in the `/etc/httpd/sites-enabled` directory. Add the example line to the bottom of your `httpd.conf` file: - - {{< file "/etc/httpd/conf/httpd.conf" apache>}} -IncludeOptional sites-enabled/*.conf - {{}} - -1. Navigate to your `/var/www/html/example.com` directory if you are not already there: - - cd /var/www/html/example.com - -1. Create the virtual host file for your website. Replace the `example.com` in `example.com.conf` with your domain name: - - sudo nano /etc/httpd/sites-available/example.com.conf - -1. Create a configuration for your virtual host. Copy the basic settings in the example below and paste them into the virtual host file you just created. Replace all instances of `example.com` with your domain name: - - {{< file "/etc/httpd/sites-availabe/example.com.conf" apache>}} - - Require all granted - - - ServerName example.com - ServerAdmin webmaster@localhost - DocumentRoot /var/www/html/example.com/public_html - ErrorLog /var/log/httpd/example.com/error.log - CustomLog /var/log/httpd/example.com/access.log combined - - order allow,deny - deny from all - - - {{}} - -1. Save the changes to the virtual host configuration file by pressing **CTRL+X** and then pressing **Y**. Press **ENTER** to confirm. - -1. Create a symbolic link from your virtual hosts file in the `sites-available` directory to the `sites-enabled` directory. Replace `example.com.conf` with the name of your own virtual hosts file. - - sudo ln -s /etc/httpd/sites-available/example.com.conf /etc/httpd/sites-enabled/example.com.conf - -1. Reload to apply your new configuration: - - sudo systemctl reload httpd - - {{< note respectIndent=false >}} -For more details on configuring your Apache virtual hosts file, see [Apache's official documentation](http://httpd.apache.org/docs/current/vhosts/). - {{< /note >}} - -1. Visit `http://example.com/wp-admin` (or `http:///wp-admin` if you haven't set up a domain) and verify that you can log in with the WordPress user you created in the [Download and Configure WordPress](#download-and-configure-wordpress) section of the guide. - -## Common Commands - -### Install Plugins - -This section covers common WP-CLI commands related to installing and updating WordPress plugins. As an example, this section will use the [Yoast SEO plugin](https://wordpress.org/plugins/wordpress-seo/). - -To install a plugin, your first step will be to find the *plugin slug*. In this case, the slug is the last part of a permalink URL which describes the plugin. If a plugin is available at http://wordpress.org/plugins/plugin-dir/, then `plugin-dir` is the slug of the plugin. You install the plugin under the same directory on your WordPress site at `http://example.com/wp-content/plugins/plugin-dir/`. Since this slug is unique to every plugin, you can search for the slug of any plugin using WP-CLI and then install it. - -1. Navigate to your WordPress site's root directory. Replace `example.com` with your own site's root directory: - - cd /var/www/html/example.com/public_html - -1. Search for the plugin you would like to install: - - wp plugin search yoast - - You will get an output similar to this. - - {{< output >}} -Success: Showing 10 of 574 plugins. -+---------------------------------------------------+------------------------------------+--------+ -| name | slug | rating | -+---------------------------------------------------+------------------------------------+--------+ -| Yoast SEO | wordpress-seo | 98 | -| Yoast SEO: Search Index Purge | yoast-seo-search-index-purge | 68 | -| ACF Content Analysis for Yoast SEO | acf-content-analysis-for-yoast-seo | 90 | -| Glue for Yoast SEO & AMP | glue-for-yoast-seo-amp | 88 | -| Google Analytics for WordPress by MonsterInsights | google-analytics-for-wordpress | 78 | -| Import Settings into WordPress SEO by Yoast | yoast-seo-settings-xml-csv-import | 100 | -| Remove Yoast SEO Comments | remove-yoast-seo-comments | 92 | -| Surbma – Yoast SEO Breadcrumb Shortcode | surbma-yoast-breadcrumb-shortcode | 84 | -| LiteSpeed Cache | litespeed-cache | 98 | -| WPGlobus – Multilingual Everything! | wpglobus | 92 | -+---------------------------------------------------+------------------------------------+--------+ -{{}} - - You can see more than 10 plugins per page by modifying the command: - - wp plugin search yoast --per-page=20 - -1. Now that you know the slug of the plugin, you want to install (`wordpress-seo`) by appending it to your command, and activate it: - - wp plugin install wordpress-seo - wp plugin activate wordpress-seo - -### Update Plugins -To update any plugin on your WordPress site: - -1. Navigate to your WordPress site's root directory. Replace `example.com` with your own site's root directory. - - cd /var/www/html/example.com/public_html - -1. Issue the plugin update command followed by the name of the plugin: - - wp plugin update wordpress-seo - - Or, to update all plugins issue the following command: - - wp plugin update --all - -1. To list all the installed plugins on your WordPress site, you can use the following command: - - wp plugin list - -### Uninstall Plugins - -To uninstall a WordPress plugin: - -1. Navigate to your WordPress site's root directory. Replace `example.com` with your own site's root directory. - - cd /var/www/html/example.com/public_html - -1. Deactivate and uninstall the plugin: - - wp plugin deactivate wordpress-seo - wp plugin uninstall wordpress-seo - -### Install Themes - -The procedure for installing and activating a theme is nearly identical to that of a plugin. Just swap `plugin` for `theme` in all of the commands. - -1. Navigate to your WordPress site's root directory. Replace `example.com` with your own site's root directory. - - cd /var/www/html/example.com/public_html - -1. Search for the theme you'd like to install. Replace `twentyfourteen` with the theme you'd like to install. - - wp theme search twentyfourteen - -1. Install and activate the theme: - - wp theme install twentyfourteen - wp theme activate twentyfourteen - -### Update Themes - -1. Navigate to your WordPress site's root directory. Replace `example.com` with your own site's root directory. - - cd /var/www/html/example.com/public_html - -1. To list all the themes in a tabular form issue the example command. You can use this command to find the theme you would like to update. - - wp theme list - -1. Update your theme: - - wp theme update twentyfourteen - - To update all themes installed on your WordPress site, issue the following command: - - wp theme update --all - -### Uninstall Themes - -1. Navigate to your WordPress site's root directory. Replace `example.com` with your own site's root directory. - - cd /var/www/html/example.com/public_html - -1. To uninstall a theme, activate a different theme first: - - wp theme activate twentyseventeen - -1. Once you've activated another theme, you can safely uninstall the previously active theme: - - wp theme uninstall twentynineteen - -### Update WordPress - -To update your WordPress site: -{{< note >}} -For more details on best practices when updating your WordPress site, see [WordPress' official documentation](https://wordpress.org/support/article/updating-wordpress/). -{{< /note >}} - -1. Navigate to your WordPress site's root directory. Replace `example.com` with your own site's root directory. - - cd /var/www/html/example.com/public_html - -1. Update all your site's files first: - - wp core update - -1. Update your site's database: - - wp core update-db - -## Next Steps - -You can now further configure WP-CLI. These commands are just the tip of the iceberg about how you can manage WordPress from the command line. Write or edit posts, perform database queries, manage user capabilities, manage `cron` events, import or export content, manage attachments, and even manage multi-site installations through a few, quick and practical keystrokes. You have refined WordPress management, and conserved valuable time. diff --git a/docs/guides/websites/ecommerce/install-magento-on-centos-7/index.md b/docs/guides/websites/ecommerce/install-magento-on-centos-7/index.md deleted file mode 100644 index a33f248ff3c..00000000000 --- a/docs/guides/websites/ecommerce/install-magento-on-centos-7/index.md +++ /dev/null @@ -1,395 +0,0 @@ ---- -slug: install-magento-on-centos-7 -title: Install Magento on CentOS 7 -description: 'Install Magento Community Edition on CentOS 7 to manage your e-commerce site.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2016-12-28 -keywords: ["magento", "centos", "e-commerce", "magento centos"] -tags: ["centos", "lamp", "cms", "ssl"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Magento Documentation](http://docs.magento.com/m2/ce/user_guide/getting-started.html)' - - '[Magento Resources Library](https://magento.com/resources)' -relations: - platform: - key: how-to-install-magento - keywords: - - distribution: CentOS 7 -aliases: [] ---- - -In this guide you'll learn how to install Magento on CentOS 7. Magento Community Edition (CE) is a free, open-source e-commerce platform. It's one of the most popular solutions for self-hosted online stores due to its simple yet powerful admin panel and large developer community. - -![Install Magento on CentOS 7](install-magento-centos-title.png "Install Magento on CentOS 7") - -Considering the resources some Magento plugins demand, we strongly recommend that you have at least a **Linode 4GB**. You'll need to allocate up to 2GB of memory for PHP and Magento to use; running Magento on a smaller Linode may result in server crashes or unreliability under medium to heavy traffic. For more memory-intensive Magento setups, we recommend using a [High Memory Linode](https://www.linode.com/pricing/). - -{{< note >}} -This guide explains how to install the latest Magento release at the time of publication. For the Community Edition, this will be version 2.1.x. If you plan to use data, themes and extensions from an older Magento site, be sure to check for compatibility issues between the two versions since not everything may function as it did in older releases. -{{< /note >}} - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's hostname and timezone. - -2. Complete the sections of our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to create a standard user account, harden SSH access and remove unnecessary network services. - -3. Magento runs on a LAMP stack, and this guide assumes you have already installed and configured Apache. If you haven't, refer to our [Apache on CentOS 7](/cloud/guides/install-and-configure-apache-on-centos-7) guide. However, do not install MariaDB or PHP. We will explain how to install compatible versions of those packages in this guide. - -3. Update your system: - - sudo yum update -{{< note >}} -This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you’re not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. -{{< /note >}} - -## Prepare Your Server for Magento - -The LAMP stack serves as the basis for a Magento site, but its component parts need to be configured a bit more before installing the Magento software. In this section, we'll explain how to add some extra settings to Apache as well as how to install MySQL 5.7 and PHP 7. - -### Configure Apache - -Since Magento will be served by Apache, some additional configuration is needed to ensure the application is served properly. In this section, we'll configure Apache for Magento to ensure that styles and other settings display properly in your storefront. - -1. Be sure you're using Apache version 2.4: - - httpd -v - - If this shows version 2.2 or another version, upgrade your packages before proceeding. - -2. Modify the virtual host file for your Magento site to resemble the example below. If you have not previously created a virtual host file, do so now and refer to our [Apache on CentOS 7](/cloud/guides/install-and-configure-apache-on-centos-7) guide for additional guidance. - - {{< file "/etc/httpd/conf.d/vhost.conf" aconf >}} - - Require all granted - - - ServerName example.com - ServerAlias www.example.com - ServerAdmin webmaster@localhost - DocumentRoot /var/www/html/example.com/public_html - - ErrorLog /var/www/html/example.com/logs/error.log - CustomLog /var/www/html/example.com/logs/access.log combined - - - Options Indexes FollowSymLinks MultiViews - AllowOverride All - - - - -{{< /file >}} - - - The `Directory` block inside the `Virtual Host` block should point to the directory where you plan to install Magento. For simplicity, we will be installing it in our web root, but if you want to put it elsewhere (e.g., a subdirectory of your web root), then modify this setting. - - It's important to note the value of `AllowOverride` as this affects which settings in each directory's `.htaccess` file will apply and which will be ignored. If you're not sure if the `All` option is optimal for your site, refer to the [Apache documentation](http://httpd.apache.org/docs/current/mod/core.html#allowoverride) for more information on this setting. - -3. Restart Apache to apply these changes: - - sudo systemctl restart httpd - -### Install MySQL and Create a Database and User - -As of this guide's publication, Magento 2 is not compatible with MariaDB, which is normally an equivalent replacement for MySQL in CentOS 7. Instead, we'll install MySQL 5.7 from a separate repository. - -1. Download and install the MySQL RPM: - - sudo wget http://dev.mysql.com/get/mysql57-community-release-el7-7.noarch.rpm -P /tmp/ - sudo yum localinstall /tmp/mysql57-community-release-el7-7.noarch.rpm - sudo yum update - sudo yum install mysql-community-server - -2. Start the MySQL daemon: - - sudo systemctl start mysqld - -3. When starting MySQL for the first time, a temporary password is generated for root access to the database. Use `grep` to find it in the log file: - - sudo grep 'temporary password' /var/log/mysqld.log - - {{< note respectIndent=false >}} -If you previously installed MariaDB or MySQL and set a root password, this installation may not override your existing database credentials. If that's the case, use the root password you set previously. -{{< /note >}} - -4. Secure your MySQL installation and change the root password with the `mysql_secure_installation` script: - - mysql_secure_installation - -5. Log into the MySQL shell as the root user, entering your new password when prompted: - - mysql -u root -p - -6. Create a Magento database and user, and set the permissions. In this example, we'll call our database and user `magento`, Replace `P@ssword1` with a secure password. You may optionally replace the other values as well: - - CREATE DATABASE magento; - CREATE USER 'magento' IDENTIFIED BY 'P@ssword1'; - GRANT ALL PRIVILEGES ON magento.* TO 'magento'; - - This section assumes that your database is hosted on the same server as your Magento application. If this is not the case, perform these steps and then refer to Magento's guide on using a [remote database server](http://devdocs.magento.com/guides/v2.0/install-gde/prereq/mysql_remote.html). - - {{< note respectIndent=false >}} -You may receive an error about the complexity of your password if it is not secure enough. By default, CentOS 7 enables the `validate-password` plugin for MySQL. For more information, refer to the official [MySQL documentation](https://dev.mysql.com/doc/refman/5.7/en/validate-password-plugin.html). - -Don't use an exclamation point (`!`) in your password. The Magento installation script does not parse them correctly in its command options. -{{< /note >}} - -7. Exit the MySQL shell: - - quit - -### Install and Configure PHP 7 - -Magento is a PHP application, so you will need to install PHP 7 and make some adjustments to its settings. - -1. CentOS 7's default repositories include PHP 5.4, which is not compatible with Magento 2. Install the *IUS* (Inline with Upstream Stable) repository to get PHP 7: - - sudo wget http://dl.iuscommunity.org/pub/ius/stable/CentOS/7/x86_64/ius-release-1.0-14.ius.centos7.noarch.rpm -P /tmp - sudo yum localinstall /tmp/ius-release-1.0-14.ius.centos7.noarch.rpm - sudo yum update - -2. Install PHP 7 and its required extensions from the IUS repository: - - sudo yum install php70u php70u-pdo php70u-mysqlnd php70u-opcache php70u-xml php70u-mcrypt php70u-gd php70u-devel php70u-mysql php70u-intl php70u-mbstring php70u-json php70u-iconv - -3. Modify the following settings in your `php.ini` file: - - {{< file "/etc/php.ini" >}} -max_input_time = 30 -memory_limit= 2G -error_reporting = E_COMPILE_ERROR|E_RECOVERABLE_ERROR|E_ERROR|E_CORE_ERROR -error_log = /var/log/php/error.log -date.timezone = America/New_York -{{< /file >}} - -This sets the time zone for PHP's `date()` function and imposes a 2GB limit to the amount of memory PHP can use. This value is recommended for a 4GB Linode, but could be increased for a larger server. - -{{< note >}} -The value for `date.timezone` will vary based on your system's time zone. Refer to the [PHP time zone documentation](http://php.net/manual/en/timezones.php) and ensure this value matches the time zone you set when you configured your Linode. -{{< /note >}} - -4. Create the log directory for PHP and give the Apache user ownership: - - sudo mkdir /var/log/php - sudo chown apache /var/log/php - -5. Because you installed a PHP module for Apache in Step 2, restart the web server to apply the changes and allow Apache to serve PHP pages: - - sudo systemctl restart httpd - -6. **Optional**: You may want to take this opportunity to create a `phpinfo.php` page to ensure that PHP is active and working properly with Apache: - - {{< file "/var/www/html/example.com/public_html/phpinfo.php" php >}} - -{{< /file >}} - -Once you've visited this page in your browser and confirmed that PHP is working, delete the test page. - -## Install Magento - -### Download Magento Community Edition - -In this section, we'll explain how to get the Magento Community Edition (CE) software on your Linode. There are several ways to do this, but here we'll cover the simplest method. - -1. On your local computer, download a copy of Magento from their [official website](https://www.magentocommerce.com/download). Several versions are available, but we recommend downloading the latest release. As of this writing, the latest version is 2.1.2: - - ![The Magento download screen](magento-download-screen.png) - - Next to your selected version, there will be a dropdown menu that says "Select your format." Choose the option ending with the `.tar.gz` extension and click **Download**. Be sure to note where you saved the downloaded file. - - ![Select the option ending in ".tar.gz"](magento-tar-gz.png) - - In this step, you'll also be able to download the software with optional sample data. Whether you choose this or the basic version is up to you. - - {{< note respectIndent=false >}} -When choosing a version, refer to Magento's [prerequisites](http://devdocs.magento.com/guides/v2.0/install-gde/prereq/prereq-overview.html) to ensure a particular version's compatibility with the components of your LAMP stack. As of this writing, Magento version 2.1.2 is compatible with all package versions covered in this guide. -{{< /note >}} - -2. From your local computer, copy the file to your Linode. Replace `/path/on/local/` with the path of your downloaded file, `user` with your standard user account on your Linode, and `yourhost` with your Linode's hostname or IP address: - - scp /path/on/local/Magento-CE-2.*.tar.gz user@yourhost:~/ - - Alternatively, you can use an FTP client, like [Filezilla](/cloud/guides/filezilla), if you're running Windows or are otherwise unable to use a command-line tool like `scp`. - -3. Log into your Linode via SSH as your standard user account. Navigate to the document root you specified in your virtual host file: - - cd /var/www/html/example.com/public_html - - Extracting the archive directly into your webroot, as you'll do in the next step, will make it accessible as the main page on your domain. For instance, if your domain is `example.com`, the Magento storefront will display when you visit `example.com` in your browser. - - If you want to run a Magento store as a subsection of your site, move and extract the archive in a directory within your webroot. For instance, if you intend to make your site accessible by visiting `example.com/store`, create the subdirectory `/var/www/html/example.com/public_html/store` and navigate to that directory before proceeding to the next step. - -4. Move the Magento archive into your document root and extract its contents: - - sudo mv ~/Magento-CE-2.*.tar.gz . - sudo tar -xvf Magento-CE-2.*.tar.gz - -### Configure Users and Permissions - -1. Create a Magento user, which will run the software. For simplicity, call this user `magento`: - - sudo useradd magento - -2. Next, add the Magento user to the web server's user group. For Apache, the default user is `apache`: - - sudo usermod -g apache magento - -3. The commands in this step should be run from your Magento installation directory (where you extracted the archive). If you are not still in that directory, navigate there before proceeding. - - Run these commands in order: - - sudo find var vendor pub/static pub/media app/etc -type f -exec chmod g+w {} \; - sudo find var vendor pub/static pub/media app/etc -type d -exec chmod g+ws {} \; - sudo chown -R magento:apache . - sudo chmod u+x bin/magento - - This allows your `magento` user (and members of the `apache` group) to write to the various files they need to run and serve Magento on your site. - - {{< note respectIndent=false >}} -The first two commands may take some time to run because they are matching various files and directories in your installation folder. It may appear that the system is inactive, but be sure to allow a couple minutes before cancelling the operations. You can always start a new SSH connection, or run these commands in a [screen](/cloud/guides/using-gnu-screen-to-manage-persistent-terminal-sessions) session, and use tools like [htop & iotop](/cloud/guides/top-htop-iotop#additional-top-like-programs) to monitor progress. -{{< /note >}} - -4. Restart Apache: - - sudo systemctl restart httpd - -### Install Magento CE - -1. Switch to the `magento` user and navigate to the `bin` directory in your Magento installation folder: - - sudo su magento - cd bin - -2. Run the Magento installation script with the following options: - - ./magento setup:install --admin-firstname="John" --admin-lastname="Doe" --admin-email="your@email.com" --admin-user="john" --admin-password="password1" --db-name="magento" --db-host="localhost" --db-user="magento" --db-password="P@ssword1" - - Replace the values in the options as follows: - - - **admin-firstname** / **admin-lastname** - This will set the full name of your admin user. Replace these with your name if you'll be the administrator. - - **admin-email** - This is your email for resetting passwords and receiving admin notifications. - - **admin-user** / **admin-password** - These are your login credentials for the Magento admin control panel. Be sure to set a secure password here and store it in a safe place. Note that the Magento script requires you to use both letter and number characters in your password, and will return an error if you do not. - - **db-name** - This is the name of the database you set up in MySQL. In our example, we called it `magento`, but if you chose a different value, substitute it here. - - **db-host** - If you're running Magento on the same server as its database, use `localhost` here. If not, this value will be the hostname of the server on which your database lives. - - **db-user** - This is the MySQL database user you set up previously. In our example, we called it `magento` but if you chose a different name, use that here instead. - - **db-password** - This will be the password you configured for the `magento` MySQL user. - - {{< note respectIndent=false >}} -These are just a few of the available options to configure your Magento installation. For more information, refer to the [Magento Installation Guide](http://devdocs.magento.com/guides/v2.1/install-gde/install/cli/install-cli-install.html) and feel free to use additional options when running the script. -{{< /note >}} - -3. The installation script may take several minutes to run. Once it's finished, you'll see a success message: - - [SUCCESS]: Magento installation complete. - [SUCCESS]: Magento Admin URI: /admin_a61e40 - -4. Exit from the `magento` user: - - exit - -Congratulations, you've successfully installed Magento on your Linode! You can log into your admin panel by entering your domain, followed by the "Magento Admin URI" displayed above, in a web browser. The **admin-user** and **admin-password** options you specified when running the installation script will be your credentials. - -## Configure Magento - -The dashboard is functional at this point, but you've still got work to do before the site is ready to use. In this section, we'll explain how to set up cron jobs and secure the Magento software to be suitable for a live e-commerce site. - -### Set Cron Jobs - -Magento relies on [cron](/cloud/guides/schedule-tasks-with-cron) to perform tasks like continuously reindexing your site and generating emails and newsletters. If you logged into your admin panel, you may have noticed an error message saying that cron jobs needed to be set. Fortunately, the cron jobs Magento uses for a base installation are easy to configure. - -1. Open the crontab for your `magento` user. Perform this step as a user with `sudo` privileges: - - sudo crontab -u magento -e - -2. If this is your first time using cron, you may be prompted to select a text editor. If you don't have a preference, select **2** to use nano. - -3. Add the following lines to the end of the file, substituting your Magento installation directory in each: - - * * * * * /bin/php /var/www/html/example.com/public_html/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/html/example.com/public_html/var/log/magento.cron.log - * * * * * /bin/php /var/www/html/example.com/public_html/update/cron.php >> /var/www/html/example.com/public_html/var/log/update.cron.log - * * * * * /bin/php /var/www/html/example.com/public_html/bin/magento setup:cron:run >> /var/www/html/example.com/public_html/var/log/setup.cron.log - - The first line handles reindexing. The second and third lines are needed by the Component Manager and System Upgrade, respectively. The `/bin/php` part of each line specifies your PHP binary. If you installed PHP using this guide, you can leave this value as we have shown it, but if you installed PHP by some other method, you can check its location with `which php` and substitute that value. - - When you're done, save and exit the file. - -4. To verify that the rules have been set properly, log out of your Magento admin and log back in. If everything has been configured correctly, you should no longer see the notification. - - If the message persists, wait several minutes before logging back in so that the cron jobs have time to take effect on the system. - -For more information about setting up cron jobs for development servers and custom Magento modules, refer to the [Magento Cron Documentation](http://devdocs.magento.com/guides/v2.1/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-bkg). - -### Configure X-Frame Options - -We strongly recommended that you disable the ability to display your Magento storefront in a frame to prevent [clickjacking](https://en.wikipedia.org/wiki/Clickjacking) attacks. To do this, modify the following line in your `env.php` file: - -{{< file "/var/www/html/example.com/public_html/app/etc/env.php" php >}} -'x-frame-options' => 'DENY', - -{{< /file >}} - - -This prevents attackers from embedding your site in a frame (for example, on a malicious site that mimics your store) in an attempt to intercept payment and other sensitive customer information. - -### Set Directory Permissions - -At a minimum, you should restrict write access to the `app/etc` directory before opening the site to customers. Run these commands from within your Magento installation directory: - - sudo find app/etc -type f -exec chmod g-w {} \; - sudo find app/etc -type d -exec chmod g-ws {} \; - -Depending on whether you install custom themes or extensions, you may need to do additional configuration. This will vary depending on what you have installed. Once you're ready to deploy your site into production mode, refer to [Magento's ownership and permissions guide](http://devdocs.magento.com/guides/v2.1/config-guide/prod/prod_file-sys-perms.html) for a more comprehensive set of recommendations. - -{{< note >}} -If you need to make additional configuration changes in the future, you'll need to manually add write permissions again before you can do so. For more information, see our guide on [Linux users and groups](/cloud/guides/linux-users-and-groups). -{{< /note >}} - -### Secure your Site with SSL - -Secure sockets layer (SSL) certificates are a vital part of e-commerce. They enable encrypted transmission of sensitive data, such as credit card numbers, that can be verified and trusted by clients. In fact, some payment vendors such as PayPal, require SSL certificates to be used for customer transactions. - -For instructions on how to use SSL certificates in your store, see our guides on [obtaining a commercially signed SSL certificate](/cloud/guides/obtain-a-commercially-signed-tls-certificate) and [using SSL certificates with Apache](/cloud/guides/ssl-apache2-centos). - -{{< note >}} -Many payment vendors that require SSL do not support self-signed certificates. Depending on how you handle payments, it is likely you will need to purchase a commercially signed certificate. - -When you [configure Apache to use the SSL certificate](/cloud/guides/ssl-apache2-centos#configure-apache-to-use-the-ssl-certificate), if you installed Magento in a subdirectory of your site and only want that section to be encrypted, make sure to modify your `` block to match. -{{< /note >}} - -Once you've installed your SSL certificate and configured Apache to serve your site securely, you'll need to configure Magento to use secure URLs. - -1. Log into your Magento admin panel using the URL you obtained after running the installation script. - -2. Click on **Stores** in the sidebar on the left of your screen, and select **Configuration** under the Settings submenu. - - ![Go to "Configuration" in the "Stores" menu.](magento-stores-config.png) - -3. On the next screen, you'll see a list of configuration settings including "General," "Catalog" and "Customers". Click on **General**, and select **Web** from its submenu. - - ![Select "Web" from the "General" menu.](magento-general-web.png) - -4. Select **Base URLs (Secure)**, which will open to display several text fields. Modify the first field, **Secure Base URL**, to use the HTTPS protocol, as opposed to HTTP: - - https://example.com - - ![Enter your domain with the HTTPS protocol.](magento-secure-base-url.png) - - This assumes that you used `/var/www/html/example.com/public_html` as your Magento installation directory, as we did in our examples. If you installed Magento in a subdirectory of your web root, you will need to modify that here. For example, if you installed Magento in a folder called `/var/www/html/example.com/public_html/magento`, you'll need to enter: - - https://example.com/magento - -5. Save your configuration. - -6. To verify that SSL has been configured correctly, visit your domain with the HTTPS protocol in a web browser and navigate to a few links. You should see a green lock icon in your browser's URL bar on each page, confirming that each page you (and your future customers) visit is secure. - -## Next Steps - -If you've followed this guide from the beginning, you should now have a secure, functional installation of Magento. You're now ready to configure your store and start selling products. - -You may wish to install extensions to add functionality, or themes to change your site's user experience. If you do, be sure that the extensions and themes you purchase or install are compatible with Magento 2. - -You may also want to configure caching with Varnish or other software to increase the speed of your site. For more information on this and other configuration options, see [Magento's configuration guide](http://devdocs.magento.com/guides/v2.1/config-guide/bk-config-guide.html). - -Finally, be sure to keep your Magento software and its components up to date. Not only is this important to the security of your site, but it will also allow you to use the latest features and functions Magento has to offer. For more information, refer to the [Magento upgrade documentation](http://devdocs.magento.com/guides/v2.1/comp-mgr/bk-compman-upgrade-guide.html). diff --git a/docs/guides/websites/ecommerce/install-magento-on-ubuntu-18-04/index.md b/docs/guides/websites/ecommerce/install-magento-on-ubuntu-18-04/index.md deleted file mode 100644 index da149d67a12..00000000000 --- a/docs/guides/websites/ecommerce/install-magento-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,430 +0,0 @@ ---- -slug: install-magento-on-ubuntu-18-04 -title: Install Magento on Ubuntu 18.04 -description: "Install Magento Community Edition on Ubuntu 18.04 to manage your e-commerce site." -authors: ["Linode"] -contributors: ["Linode"] -published: 2016-12-28 -modified: 2018-08-29 -keywords: ["magento", "ubuntu", "e-commerce", "magento ubuntu"] -tags: ["ubuntu", "ssl", "lamp", "cms","ecommerce"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -external_resources: - - '[Magento Documentation](http://docs.magento.com/m2/ce/user_guide/getting-started.html)' - - '[Magento Resources Library](https://magento.com/resources)' -relations: - platform: - key: how-to-install-magento - keywords: - - distribution: Ubuntu 18.04 -aliases: [] ---- - -In this guide you'll learn how to install Magento on Ubuntu 18.04\. Magento Community Edition (CE) is a free, open-source e-commerce platform. It's one of the most popular solutions for self-hosted online stores due to its simple yet powerful admin panel and large developer community. - -Considering the resources some Magento plugins demand, we strongly recommend that you have at least a **Linode 4GB**. You'll need to allocate up to 2GB of memory for PHP and Magento; running Magento on a smaller Linode may result in server crashes or unreliability under medium to heavy traffic. For more memory-intensive Magento setups, we recommend using a [High Memory Linode](https://www.linode.com/pricing/). - -{{< note >}} This guide explains how to install the latest Magento release at the time of publication. For the Community Edition, this will be version 2.2.x. If you plan to use data, themes, and extensions from an older Magento site, be sure to check for compatibility issues between the two versions. {{< /note >}} - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's hostname and timezone. - -2. Complete the sections of our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to create a standard user account, harden SSH access, and remove unnecessary network services. - -3. Magento runs on a LAMP stack, and is **only compatible with PHP 7.0 or 7.1**. This guide contains instructions for setting up Apache, MySQL, and PHP in a manner compatible with the latest version of Magento (v. 2.2.5). Check your version of PHP with `php -v`. If your version is incompatible, refer to the included instructions for installing and setting PHP 7.0 as the default version. - - {{< note respectIndent=false >}} If you have an existing LAMP stack installed on your Linode, refer to Magento's [prerequisites](https://devdocs.magento.com/guides/v2.2/install-gde/prereq/prereq-overview.html) to ensure a particular version's compatibility with the components of your stack. {{< /note >}} - -4. Update your system: - - ``` - sudo apt update && sudo apt upgrade - ``` - - {{< note respectIndent=false >}} This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/cloud/guides/linux-users-and-groups) guide. {{< /note >}} - -## Install A Magento-compatible LAMP Stack - -1. Install Apache 2.4. - - ``` - apt install apache2 - systemctl enable apache2 - ``` - -2. Install MariaDB 10.2 - - {{< note respectIndent=false >}} Magento is also compatible with MySQL 5.7.x, MySQL NDB Cluster 7.4.x, MariaDB 10.0, 10.1, 10.2, Percona 5.7 and other binary compatible MySQL technologies. You may choose to use any one of the listed versions. {{< /note >}} - - ``` - sudo apt-get install software-properties-common - sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8 - sudo add-apt-repository 'deb [arch=amd64] http://mariadb.biz.net.id/repo/10.2/ubuntu bionic main' - ``` - - Update packages and install MariaDB. - - ``` - apt update - apt install mariadb-server - ``` - - Setup MariaDB using the mysql_secure_installation built-in shell script. Supply each prompt with `y` and hit `Enter`. Set a secure password for the *root* user when prompted. - - ``` - mysql_secure_installation - ``` - - Set MariaDB to start on system boot and initialize the daemon. - - ``` - systemctl enable mysql - systemctl start mysql - ``` - -3. Install the PHP 7.0 repository and update packages. - - ``` - sudo add-apt-repository ppa:ondrej/php - sudo apt update -y - ``` - - Install all required PHP extensions. - - ``` - sudo apt install -y php7.0 libapache2-mod-php7.0 php7.0-common php7.0-gd php7.0-mysql php7.0-mcrypt php7.0-curl php7.0-intl php7.0-xsl php7.0-mbstring php7.0-zip php7.0-bcmath php7.0-iconv php7.0-soap - ``` - -4. **Optional** - Proceed with this step only if you already have an incompatible PHP version installed and need to switch. - - Replace `php7.x` with your incompatible PHP version in the first command. - - ``` - sudo a2dismod php7.x && sudo a2enmod php7.0 - sudo systemctl restart apache2 - sudo update-alternatives --set php /usr/bin/php7.0 - ``` - - Confirm your version of PHP with `php -v` and verify that the output matches the following: - - ``` - PHP 7.0.31-1+ubuntu18.04.1+deb.sury.org+1 (cli) (built: Jul 25 2018 10:01:10) ( NTS ) - Copyright (c) 1997-2017 The PHP Group - Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies - with Zend OPcache v7.0.31-1+ubuntu18.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies - ``` - -## Configure Apache - -Since Magento will be served by Apache, some additional configuration is needed to ensure the application is served properly. In this section, we'll configure Apache for Magento to ensure that styles and other settings display properly in your storefront. - -1. Confirm you're using Apache version 2.4: - - ``` - apache2 -v - ``` - - If this shows version 2.2 or another version, upgrade Apache before proceeding. - -2. Enable the Apache rewrite module: - - ``` - sudo a2enmod rewrite - ``` - -3. Modify the virtual host file for your Magento site to resemble the example below. If you have not previously created a virtual host file, do so now and refer to the [Configure Virtual Hosts](/cloud/guides/install-lamp-stack-on-ubuntu-16-04#configure-virtual-hosts) section of the LAMP on Ubuntu 16.04 guide for additional guidance. - - {{< file "/etc/apache2/sites-available/example.com.conf" apache >}} - - - Require all granted - - - - ServerName example.com - ServerAlias www.example.com - ServerAdmin webmaster@localhost - DocumentRoot /var/www/html/example.com/public_html - - ErrorLog /var/www/html/example.com/logs/error.log - CustomLog /var/www/html/example.com/logs/access.log combined - - - Options Indexes FollowSymLinks MultiViews - AllowOverride All - - - -{{< /file >}} - - The `Directory` block inside the `Virtual Host` block should point to the directory where you plan to install Magento. For simplicity, we will be installing it in our web root, but if you want to put it elsewhere (e.g., a subdirectory of your web root), modify this setting. - - It's important to note the value of `AllowOverride` as this affects which settings in each directory's `.htaccess` file will be applied and which will be ignored. If you're not sure if the `All` option is optimal for your site, refer to the [Apache documentation](http://httpd.apache.org/docs/current/mod/core.html#allowoverride) for more information on this setting. - -4. If you haven't already, enable your site: - - ``` - sudo a2ensite example.com - ``` - - Replace `example.com` with the name of your site's virtual host. - -5. Restart Apache to apply these changes: - - ``` - sudo systemctl restart apache2 - ``` - -6. **Optional**: You may want to take this opportunity to create a `phpinfo.php` page to ensure that PHP is active and working properly with Apache: - - {{< file "/var/www/html/example.com/public_html/phpinfo.php" php >}}{{< /file >}} - - Once you've visited this page in your browser and confirmed that PHP is working, delete the test page. - -## Create a MySQL Database and User - -If you previously installed a LAMP stack using this or our other guides you should already have set up MySQL root login credentials. If not, you'll need to go back and do so before proceeding. - -1. Log into the MySQL shell as the root user, entering your password when prompted: - - ``` - mysql -u root -p - ``` - -2. Create a Magento database and user, and set the permissions. In this example, we'll call our database and user `magento`. Replace `password` with a secure password. You may optionally replace the other values as well: - - ``` - CREATE DATABASE magento; - CREATE USER 'magento' IDENTIFIED BY 'password'; - GRANT ALL PRIVILEGES ON magento.* TO 'magento'; - ``` - - This section assumes that your database is hosted on the same server as your Magento application. If this is not the case, perform these steps and then refer to Magento's guide on using a [remote database server](https://devdocs.magento.com/guides/v2.2/install-gde/prereq/mysql_remote.html). - -3. Exit the MySQL shell: - - ``` - quit - ``` - -## Install Magento - -### Download Magento Community Edition - -In this section, we'll explain how to install the Magento Community Edition (CE) software on your Linode. There are several ways to do this, but here we'll cover the simplest method. Before starting this section, you must create an account to gain access to the Magento software. Create an account [here](https://account.magento.com/customer/account/create/). - -1. On your local computer, download a copy of Magento from their [official website](https://www.magentocommerce.com/download). Several versions are available, but we recommend downloading the latest release. As of this writing, the latest version is 2.2.5. - - ![The Magento download screen](magento-download-screen.png) - - Next to your selected version, there will be a dropdown menu that says "Select your format." Select the option ending with the `.tar.gz` extension and click the **Download** button. Note that in this step, you will be presented with the option to download the software with optional sample data. Whether you choose this or the basic version is up to you. - - ![Select the option ending in ".tar.gz"](magento-tar-gz.png) - -2. From your local computer, copy the file to your Linode. Replace */path/on/local/* with the path of your downloaded file, *user* with your standard user account on your Linode, and *host-ip_or_web-url* with your Linode’s web-url or IP address: - - ``` - scp /path/on/local/Magento-CE-2.*.tar.gz user@host-ip_or_web-url:~/ - ``` - - Alternatively, you can use an FTP client, like [Filezilla](/cloud/guides/filezilla), if you're running Windows or are otherwise unable to use a command-line tool like `scp`. - -3. Log into your Linode via SSH as your standard user account. Navigate to the document root you specified in your virtual host file: - - ``` - cd /var/www/html/example.com/public_html - ``` - - Extracting the archive directly into your webroot, as you'll do in the next step, will make it accessible as the main page on your domain. For instance, if your domain is `example.com`, the Magento storefront will display when you visit `example.com` in your browser. - - If you want to run a Magento store as a subsection of your site, move and extract the archive in a directory within your webroot. For instance, if you intend to make your site accessible by visiting `example.com/store`, create the subdirectory `/var/www/html/example.com/public_html/store` and navigate to that directory before proceeding to the next step. - -4. Move the Magento archive into your document root and extract its contents: - - ``` - sudo mv ~/Magento-CE-2.*.tar.gz . - sudo tar -xvf Magento-CE-2.*.tar.gz - ``` - -## Configure Users and Permissions - -1. Create a Magento user, which will run the software. For simplicity, call this user `magento`: - - ``` - sudo useradd magento - ``` - -2. Next, add the Magento user to the web server's user group. For Apache, the default user is `www-data`: - - ``` - sudo usermod -g www-data magento - ``` - -3. The commands in this step should be run from your Magento installation directory (where you extracted the archive). If you are not still in that directory, navigate there before proceeding. - - Run these commands in order: - - ``` - sudo find var vendor pub/static pub/media app/etc -type f -exec chmod g+w {} \; - sudo find var vendor pub/static pub/media app/etc -type d -exec chmod g+ws {} \; - sudo chown -R magento:www-data . - sudo chmod u+x bin/magento - ``` - - This allows your `magento` user (and members of the `www-data` group) to write to the various files they need to run and serve Magento on your site. - - {{< note respectIndent=false >}} The first two commands may take some time to run because they are matching various files and directories in your installation folder. It may appear that the system is inactive, but be sure to allow a couple of minutes before cancelling the operations. You can always start a new SSH connection, or run these commands in a [screen](/cloud/guides/using-gnu-screen-to-manage-persistent-terminal-sessions) session, and use tools like [htop & iotop](/cloud/guides/top-htop-iotop#additional-top-like-programs) to monitor progress. {{< /note >}} - -4. Restart Apache: - - ``` - sudo systemctl restart apache2 - ``` - -## Install Magento CE - -1. Switch to the `magento` user and navigate to the `bin` directory in your Magento installation folder: - - ``` - sudo su magento - cd bin - ``` - -2. Run the Magento installation script with the following options: - - - ``` - ./magento setup:install --admin-firstname="John" --admin-lastname="Doe" --admin-email="your@email.com" --admin-user="john" --admin-password="password1" --db-name="magento" --db-host="localhost" --db-user="magento" --db-password="password" - ``` - - Replace the values in the options as follows: - - - **admin-firstname** / **admin-lastname** - This will set the full name of your admin user. Replace these with your name if you'll be the administrator. - - **admin-email** - This is your email for resetting passwords and receiving admin notifications. - - **admin-user** / **admin-password** - These are your login credentials for the Magento admin control panel. Be sure to set a secure password here and store it in a safe place. Note that the Magento script requires you to use both letter and number characters in your password, and will return an error if you do not. - - **db-name** - This is the name of the database you set up in MySQL. In our example, we called it `magento`, but if you chose a different value, substitute it here. - - **db-host** - If you're running Magento on the same server as its database, use `localhost` here. If not, this value will be the hostname of the server on which your database lives. - - **db-user** - This is the MySQL database user you set up previously. In our example, we called it `magento` but if you chose a different name, use that here instead. - - **db-password** - This will be the password you configured for the `magento` MySQL user. - - {{< note respectIndent=false >}} These are just a few of the available options to configure your Magento installation. For more information, refer to the [Magento Installation Guide](http://devdocs.magento.com/guides/v2.1/install-gde/install/cli/install-cli-install.html) and feel free to use additional options when running the script. {{< /note >}} - -3. The installation script may take several minutes to run. Once it's finished, you'll see a success message: - - ``` - [SUCCESS]: Magento installation complete. [SUCCESS]: Magento Admin URI: /admin_a61e40 - ``` - - You've successfully installed Magento on your Linode. You can log into your admin panel by entering your domain, followed by the "Magento Admin URI" displayed above, in a web browser. The **admin-user** and **admin-password** options you specified when running the installation script will be your credentials. - -4. Exit from the `magento` user: - - ``` - exit - ``` - -## Configure Magento - -The dashboard is functional at this point, but you've still got work to do before the site is ready to use. In this section, we'll explain how to set up cron jobs and secure the Magento software to be suitable for a live e-commerce site. - -### Set Cron Jobs - -Magento relies on [cron](/cloud/guides/schedule-tasks-with-cron) to perform tasks like continuously reindexing your site and generating emails and newsletters. If you logged into your admin panel, you may have noticed an error message saying that cron jobs needed to be set. Fortunately, the cron jobs Magento uses for a base installation are very easy to configure. - -1. Open the crontab for your `magento` user. Perform this step as a user with `sudo` privileges: - - ``` - sudo crontab -u magento -e - ``` - -2. If this is your first time using cron, you'll be prompted to select a text editor. If you don't have a preference, select **2** to use nano. - -3. Add the following lines to the end of the file, substituting your Magento installation directory in each: - - * * * * * /usr/bin/php /var/www/html/example.com/public_html/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/html/example.com/public_html/var/log/magento.cron.log - - * * * * * /usr/bin/php /var/www/html/example.com/public_html/update/cron.php >> /var/www/html/example.com/public_html/var/log/update.cron.log - - * * * * * /usr/bin/php /var/www/html/example.com/public_html/bin/magento setup:cron:run >> /var/www/html/example.com/public_html/var/log/setup.cron.log - - The first line handles reindexing. The second and third lines are needed by the Component Manager and System Upgrade, respectively. The `/usr/bin/php` part of each line specifies your PHP binary. If you installed PHP using our LAMP stack guide, you can leave this value as we have shown it, but if you installed PHP by some other method, you can check its location with `which php` and substitute that value. - - When you're done, save and exit the file. - -To verify that the rules have been set properly, log out of your Magento admin and log back in. If everything has been configured correctly, you should no longer see the notification. - -For more information about setting up cron jobs for development servers and custom Magento modules, refer to the [Magento Cron Documentation](http://devdocs.magento.com/guides/v2.1/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-bkg). - -### Configure X-Frame Options - -We strongly recommend disabling the ability to display your Magento storefront in a frame to prevent [clickjacking](https://en.wikipedia.org/wiki/Clickjacking) attacks. To do this, modify the following line in your `env.php` file: - -{{< file "/var/www/html/example.com/public_html/app/etc/env.php" php >}} 'x-frame-options' => 'DENY', - -{{< /file >}} - -This prevents attackers from embedding your site in a frame (for example, on a malicious site that mimics your store) in an attempt to intercept payment and other sensitive customer information. - -### Set Directory Permissions - -At a minimum, you should restrict write access to the `app/etc` directory before opening the site to customers. Run these commands from within your Magento installation directory: - -``` -sudo find app/etc -type f -exec chmod g-w {} \; -sudo find app/etc -type d -exec chmod g-ws {} \; -``` - -Depending on whether you install custom themes or extensions, you may need to do additional configuration. This will vary depending on what you have installed. Once you're ready to deploy your site into production mode, refer to [Magento's ownership and permissions guide](https://devdocs.magento.com/guides/v2.2/config-guide/prod/prod_file-sys-perms.html) for a more comprehensive set of recommendations. - -{{< note >}} If you need to make additional configuration changes in the future, you'll need to manually add write permissions again before you can do so. For more information, see our guide on [Linux users and groups](/cloud/guides/linux-users-and-groups). {{< /note >}} - -### Secure your Site with SSL - -Secure sockets layer (SSL) certificates are a vital part of e-commerce. They enable encrypted transmission of sensitive data, such as credit card numbers, that can be verified and trusted by clients. In fact, some payment vendors such as PayPal, require SSL certificates to be used for customer transactions. - -For instructions on how to use SSL certificates in your store, see our guides on [obtaining a commercially signed SSL certificate](/cloud/guides/obtain-a-commercially-signed-tls-certificate) and [using SSL certificates with Apache](/cloud/guides/ssl-apache2-debian-ubuntu). - -{{< note >}} Many payment vendors that require SSL do not support self-signed certificates. Depending on how you handle payments, it is likely you will need to purchase a commercially signed certificate. - -When you [configure Apache to use the SSL certificate](/cloud/guides/ssl-apache2-centos#configure-apache-to-use-the-ssl-certificate), if you installed Magento in a subdirectory of your site, and only want that section to be encrypted, make sure to modify your `` block to match. {{< /note >}} - -Once you've installed your SSL certificate and configured Apache to serve your site securely, you'll need to configure Magento to use secure URLs. - -1. Log into your Magento admin panel using the URL you obtained after running the installation script. - -2. Click on **Stores** in the sidebar on the left of your screen, and select **Configuration** under the Settings submenu. - - ![Go to "Configuration" in the "Stores" menu.](magento-stores-config.png) - -3. On the next screen, you'll see a list of configuration settings including "General," "Catalog," and "Customers". Click on **General**, and select **Web** from its submenu. - - ![Select "Web" from the "General" menu.](magento-general-web.png) - -4. Select **Base URLs (Secure)**, which will open to display several text fields. Modify the first field, **Secure Base URL**, to use the HTTPS protocol, as opposed to HTTP: - - ``` - https://example.com - ``` - - ![Enter your domain with the HTTPS protocol.](magento-secure-base-url.png) - - This assumes that you used `/var/www/html/example.com/public_html` as your Magento installation directory, as we did in our examples. If you installed Magento in a subdirectory of your web root, you will need to modify that here. For example, if you installed Magento in a folder called `/var/www/html/example.com/public_html/magento`, you'll need to enter: - - ``` - https://example.com/magento - ``` - -5. Save your configuration. - -6. To verify that SSL has been configured correctly, visit your domain with the HTTPS protocol in a web browser and navigate to a few links. You should see a green lock icon in your browser's URL bar on each page, confirming that each page you (and your future customers) visit is secure. - -## Next Steps - -If you've followed this guide from the beginning, you should now have a secure, functional installation of Magento. You're now ready to configure your store and start selling products. - -You may wish to install extensions to add functionality, or themes to change your site's user experience. If you do, be sure that the extensions and themes you purchase or install are compatible with Magento 2. - -You may also want to configure caching with Varnish or other software to increase the speed of your site. For more information on this and other configuration options, see [Magento's configuration guide](https://devdocs.magento.com/guides/v2.2/config-guide/bk-config-guide.html). - -Finally, be sure to keep your Magento software and its components up to date. Not only is this important to the security of your site, but it will also allow you to use the latest features and functions Magento has to offer. For more information, refer to the [Magento upgrade documentation](http://devdocs.magento.com/guides/v2.1/comp-mgr/bk-compman-upgrade-guide.html). diff --git a/docs/guides/websites/ecommerce/install-opencart-on-centos-7/index.md b/docs/guides/websites/ecommerce/install-opencart-on-centos-7/index.md deleted file mode 100644 index c2eab5de2da..00000000000 --- a/docs/guides/websites/ecommerce/install-opencart-on-centos-7/index.md +++ /dev/null @@ -1,156 +0,0 @@ ---- -slug: install-opencart-on-centos-7 -title: How to Install OpenCart on CentOS 7 -description: 'This tutorial will guide you through basic installation and set-up of popular ecommerce platform OpenCart on CentOS 7.' -authors: ["Jonathan Chun"] -contributors: ["Jonathan Chun"] -published: 2017-04-29 -modified: 2017-08-21 -keywords: ["opencart", "ecommerce", "commerce", "store", "centos"] -tags: ["centos"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[OpenCart Website](https://www.opencart.com)' - - '[OpenCart Documentation](http://docs.opencart.com/)' - - '[OpenCart Community Forums](http://forum.opencart.com/)' -relations: - platform: - key: how-to-install-opencart - keywords: - - distribution: CentOS 7 ---- - -![Header image](How_to_Install_OpenCart_on_CentOS_smg.jpg "How to Install OpenCart on CentOS 7") - -## What is OpenCart? - -[OpenCart](https://www.opencart.com/) is free open source ecommerce platform for online merchants. OpenCart provides a professional and reliable foundation from which to build a successful online store. This foundation appeals to a wide variety of users; ranging from seasoned web developers looking for a user-friendly interface to use, to shop owners just launching their business online for the first time. OpenCart has an extensive amount of features that gives you a strong hold over the customization of your store. With OpenCart's tools, you can help your online shop live up to its fullest potential. - -## Before You Begin - -1. You should set up [LAMP on CentOS 7](/cloud/guides/how-to-install-a-lamp-stack-on-centos-7). - -2. Install the Extra Packages for Enterprise Linux (EPEL) repository. - sudo yum install epel-release - -3. Make sure your server packages are up to date. - sudo yum update - -4. Install the unzip utility. - sudo yum install unzip - - -## PHP Setup - -Make sure that you have the following PHP extensions/modules installed (They should be installed by default on almost all PHP installations): -- Curl -- Zip -- Zlib -- GD Library -- Mcrypt -- Mbstrings -- Xml - -You can install them by running the following commands: - - sudo yum install php-mcrypt curl zlib php-gd php-mbstring php-xml php-mysql - -## Create a Database and User - -You will need to make sure you have a MySQL database and a database user set up for OpenCart. To create a database and a database user, please complete the following instructions: - -1. Login to MySQL/MariaDB: - - mysql -u root -p - - Enter your MySQL/MariaDB root password when prompted. You should now see a MySQL prompt similar to - - MariaDB [(none)]> - -2. Create a new database and user with permissions to use it for OpenCart: - - create database opencart; - grant all on opencart.* to 'ocuser' identified by 'yourpassword'; - flush privileges; - - In the above example, `opencart` is the name of the database, `ocuser` the user, and `yourpassword` a strong password. - -3. Exit MySQL/MariaDB. - - quit - -## Install OpenCart - -### Download OpenCart and configure permissions - -Issue the following commands to download and unpack OpenCart into your web root directory: - - cd /var/www/html/example.com/public_html/ - curl -OL https://github.com/opencart/opencart/releases/download/2.3.0.2/2.3.0.2-compiled.zip - unzip 2.3.0.2-compiled.zip 'upload/*' - cp upload/config-dist.php upload/config.php - cp upload/admin/config-dist.php upload/admin/config.php - sudo chown -R apache:apache upload - shopt -s dotglob - mv upload/* . - rm -rf upload - -In the above example, replace `/var/www/html/example.com/public_html/` with your virtual host's web root and `2.3.0.2-compiled.zip` with the file name of the latest OpenCart release. - -{{< note >}} -You can find the latest version of OpenCart from their [Download Page](https://www.opencart.com/?route=cms/download). -{{< /note >}} - -Next, run the following commands to make sure these directories are writable. - - chmod 0755 system/storage/cache/ - chmod 0755 system/storage/logs/ - chmod 0755 system/storage/download/ - chmod 0755 system/storage/upload/ - chmod 0755 system/storage/modification/ - chmod 0755 image/ - chmod 0755 image/cache/ - chmod 0755 image/catalog/ - chmod 0755 config.php - chmod 0755 admin/config.php - -### Run the OpenCart Installer - -Visit your website in your browser. You should be redirected to the OpenCart Installer in a page that looks like this: - -![OpenCart Installer Page 1.](opencart-1.png) - -Press **CONTINUE** and you will see the _Pre-Installation_ page. - -![OpenCart Installer Page 2.](opencart-2.png) - -Make sure that you see the green checkmark for every section before proceeding. If you see any red marks, please go back and make sure all of the instructions in this guide were followed properly. Once you've confirmed that every section has a green checkmark, press the **CONTINUE** button to see the OpenCart configuration page: - -![OpenCart Installer Page 3.](opencart-3.png) - -Here, select the **MySQLi** driver and fill out the form with the relevant information from the [Create a Database and User](#create-a-database-and-user) section of this guide. You also need to select a username and password for the default administrator account for this OpenCart installation. - -Finally, press the **CONTINUE** button and you should now see the _**Installation complete**_ screen. You will also see a warning box telling you to remove your installation directory. We can do this by going back to your web root directory and running: - - rm -rf install - -Once you've done that, the installation is complete. Browse to - - http://example.com/admin - -In the above example, `example.com` is your domain name. - -You can now login using the administrator account details you entered previously. - -## Recommended Next Steps - -Now that you have your OpenCart installation up and running, there are a few more steps that we recommend. Because of the security-sensitive nature of ecommerce websites, you will want to make sure your system is secure. - -1. Make sure you follow our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide. - -2. Secure your website with SSL/TLS. Learn to [Install a SSL certificate with Apache on CentOS 7](/cloud/guides/ssl-apache2-centos). Once you've installed a certificate, enable **Use SSL** by following the [Opencart Documentation on SSL](http://docs.opencart.com/administration/ssl/). - -3. Follow the [Basic Security Practices](http://docs.opencart.com/administration/security/) from the OpenCart documentation. - -4. Install [vQmod](https://github.com/vqmod/vqmod) if you wish to use 3rd party extensions. diff --git a/docs/guides/websites/erp/install-an-odoo-13-stack-on-ubuntu-18-04/index.md b/docs/guides/websites/erp/install-an-odoo-13-stack-on-ubuntu-18-04/index.md deleted file mode 100644 index f56e4b92425..00000000000 --- a/docs/guides/websites/erp/install-an-odoo-13-stack-on-ubuntu-18-04/index.md +++ /dev/null @@ -1,392 +0,0 @@ ---- -slug: install-an-odoo-13-stack-on-ubuntu-18-04 -title: Installing an Odoo 13 Stack on Ubuntu 18.04 -title_meta: 'How to Install an Odoo 13 Stack on Ubuntu 18.04' -description: 'Odoo is an open-source suite of over 10,000 business apps with a web interface for managing them. This guide shows how to install Odoo 13 on Ubuntu 18.04.' -authors: ["Damaso Sanoja"] -contributors: ["Damaso Sanoja"] -published: 2020-09-17 -keywords: ["Odoo 13 install ubuntu 18.04", "install open source cms erp ubuntu"] -tags: ["debian", "postgresql", "database", "cms", "digital agencies"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -image: InstallOdoo13Stack_Ubuntu1804.png -aliases: [] -external_resources: - - '[Odoo User Documentation](https://www.odoo.com/documentation/user/13.0/)' - - '[Odoo Developer Documentation](https://www.odoo.com/documentation/13.0)' - - '[PostgreSQL 10 Documentation](https://www.postgresql.org/docs/10/static/index.html)' - - '[Install an Odoo 11 Stack on Ubuntu 16.04 using Linode](/cloud/guides/install-an-odoo-11-stack-on-ubuntu-16-04)' - - '[Install an SSL certificate with LetsEncrypt](/cloud/guides/install-lets-encrypt-to-create-ssl-certificates)' - - '[How to Set up tinc, a Peer-to-Peer VPN](/cloud/guides/how-to-set-up-tinc-peer-to-peer-vpn)' - - '[Using Terraform to Provision Linode Environments](/cloud/guides/how-to-build-your-infrastructure-using-terraform-and-linode)' -relations: - platform: - key: install-an-odoo-13-stack - keywords: - - distribution: Ubuntu 18.04 ---- - -## What is Odoo? - -[Odoo](https://www.odoo.com/) (formerly known as OpenERP) is a self-hosted suite of over 10,000 open source applications for a variety of business needs, including CRM, eCommerce, accounting, inventory, point of sale, and project management. These applications are all fully integrated and can be installed and accessed through a web interface, making it easy to automate and manage your company's processes. - -For simple installations, Odoo and its dependencies can be installed on a single Linode (see our [Install Odoo 10 on Ubuntu 16.04](/cloud/guides/install-odoo-10-on-ubuntu-16-04) guide for an example of this). However, this single-server setup is not suited for production deployments. This guide covers how to configure a production Odoo 13 cluster where the Odoo server and PostgreSQL database are hosted on separate Linodes. This configuration gives you more flexibility and scalability while allowing you to use PostgreSQL database replication for added performance and reliability. - -## System Requirements - -The setup in this guide requires the following *minimal* Linode specifications: - -* A Shared **2GB** Linode to install the PostgreSQL 10 database -* A Shared **1GB** Linode (Nanode) to install the Odoo 13 web application - -Keep in mind that your implementation may need more nodes or higher-memory plans depending on the number of end-users you want to serve and the number of modules you plan to incorporate. - -All examples in this guide are for Ubuntu 18.04. If you plan to use a different operating system, adapt the commands as necessary. - -## Before You Begin - -1. Familiarize yourself with our [Getting Started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide and complete the steps for setting your Linode's hostname and timezone. - -1. This guide uses `sudo` wherever possible. Complete the sections of our [Securing Your Server](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) to create a standard user account, harden SSH access, and remove unnecessary network services. - -1. Update your systems: - - sudo apt-get update && sudo apt-get upgrade - -## Configure Firewall Rules for Odoo - -If you want to configure a firewall for your Linodes, open the following ports: - -| Node | Open TCP Ports | -| ------------ |:--------:| -| Odoo 13 application | `22, 6010, 5432, 8069` | -| PostgreSQL database | `22, 6010, 5432` | - -Ports `22`, `80`, and `5432` are the defaults for SSH, HTTP, and PostgreSQL communications, respectively. Port `6010` is used for Odoo communications and port `8069` is used by Odoo's webserver. To open a particular port, use a command similar to the following: - - sudo ufw allow 22/tcp - -For more detailed information about firewall setup please read our guide [How to Configure a Firewall with UFW](/cloud/guides/configure-firewall-with-ufw). - -## Hostname Assignment - -In order to simplify communication between Linodes, set hostnames for each server. You can use private IPs if the Linodes are all in the same data center, or Fully Qualified Domain Names (FQDNs) if available. This guide uses the following FQDN and hostname conventions: - -| Node | Hostname | FQDN | -| ------------ |:--------:| :-----------:| -| Odoo 13 | odoo | odoo.yourdomain.com | -| PostgreSQL | postgresql | postgresql.yourdomain.com | - -- PostgreSQL server: - - {{< file "/etc/hosts" conf >}} -127.0.0.1 localhost -127.0.1.1 postgresql.yourdomain.com postgresql - -10.1.3.10 odoo.yourdomain.com odoo - -{{< /file >}} - -- Odoo 13 server: - - {{< file "/etc/hosts" conf >}} -127.0.0.1 localhost -127.0.1.1 odoo.yourdomain.com odoo - -10.1.1.10 postgresql.yourdomain.com postgresql -{{< /file >}} - -FQDNs are used throughout this guide whenever possible to avoid confusion. - -## Set up PostgreSQL Database - -Configure Odoo's `postgresql` database backend Linode. The Ubuntu 18.04 official repository includes PostgreSQL version 10 which offers significant performance improvements as well as database replication compatibility. - -Install the PostgreSQL database and developer libraries with the following command: - - sudo apt install postgresql-10 postgresql-server-dev-10 -y - -### Create PostgreSQL User - -Odoo requires a separate PostgreSQL user for communications between the web application Linode and the database Linode. Create the database user `odoo`. This user is in charge of all operations. Use a strong password and save it in a secure location to use later: - - sudo -u postgres createuser odoo -U postgres -dP - -The options used are described below: - -* `-u`: Executes the command as the `postgres` user. -* `-U`: Indicates the user name to connect as. -* `-d`: Grants the user permission to create databases. -* `-P`: Prompts you for the new user's password. - -### Configure Host Based Authentication - -1. Stop the PostgreSQL service: - - sudo systemctl stop postgresql - -1. Edit the `pg_hba.conf` file to allow PostgreSQL Linode to communicate with the Odoo Linode server. Add the following line to the file: - - {{< file "/etc/postgresql/10/main/pg_hba.conf" conf >}} -host all odoo odoo.yourdomain.com md5 -{{< /file >}} - -This line grants the `odoo` user the rights connect to `all` databases within this server. - -The settings in the `pg_hba.conf` file are: - -* `host`: Enables connections using Unix-domain sockets. -* `all`: Match all databases on the server. You can provide a comma separated list of specific Odoo database names if you know them beforehand. -* `odoo`: The Odoo user responsible for application/database communications. -* `odoo.yourdomain.com`: The address of your Odoo server. You should replace this with your FQDN or Ip address. -* `md5`: Make use of client-supplied MD5-encrypted passwords for authentication. - -### Configure PostgreSQL listening address - -Edit `postgresql.conf` to allow the database server listening to remote connections: - -{{< file "/etc/postgresql/10/main/postgresql.conf" conf >}} -#From CONNECTIONS AND AUTHENTICATION Section -listen_addresses = '*' -{{< /file >}} - -These settings are: - -* `listen_addresses`: What IP addresses to listen on. The `'*'` means that the server listens to all IP addresses. You can limit this to only include the IP addresses that you consider safe. - -### Enable PostgreSQL on Startup - -Now that you finished PostgreSQL configuration you can start the `postgresql` service and enable it on startup: - - sudo systemctl start postgresql && sudo systemctl enable postgresql - -## Odoo 13 Setup - -Configure your Odoo 13 web application to work with the PostgreSQL database backend. - -{{< note >}} -Odoo 13 uses Python 3.6+ instead of Python 3.5. If your server is running an older Ubuntu release, for instance 16.04, you will need to compile a newer Python version to meet this requirement. -{{< /note >}} - -### Prepare Linode for Odoo 13 Installation - -1. In order to separate Odoo from other services, create a new Odoo system user to run its processes: - - sudo adduser --system --home=/opt/odoo --group odoo - -1. Install system dependencies that are needed during Odoo 13 set up: - - sudo apt-get install python3 python3-pip python3-suds python3-all-dev python3-venv \ - python3-dev python3-setuptools python3-tk libxml2-dev libxslt1-dev libevent-dev \ - libsasl2-dev libldap2-dev pkg-config libtiff5-dev libjpeg8-dev libjpeg-dev \ - zlib1g-dev libfreetype6-dev liblcms2-dev liblcms2-utils libwebp-dev tcl8.6-dev \ - tk8.6-dev libyaml-dev fontconfig xfonts-75dpi xfonts-base xfonts-encodings xfonts-utils -y - -1. Use Git to clone the Odoo files onto your server: - - sudo git clone https://www.github.com/odoo/odoo.git --depth 1 \ - --branch 13.0 --single-branch /opt/odoo - -1. Enforce the use of POSIX locale to prevent possible errors during installation (this has nothing to do with the Odoo language): - - export LC_ALL=C - -1. Install Less CSS via Node.js and npm: - - sudo curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash - \ - && sudo apt install -y nodejs \ - && sudo npm install -g less less-plugin-clean-css - -1. Download `wkhtmltopdf` version `0.12.5` which is the recommended version for Odoo 13. For more information regarding `wkhtmltopdf` recommended versions, visit [Odoo wiki](https://github.com/odoo/odoo/wiki/Wkhtmltopdf) - - cd /tmp - wget https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.5/wkhtmltox_0.12.5-1.bionic_amd64.deb - -1. Install the package: - - sudo dpkg -i wkhtmltox_0.12.5-1.bionic_amd64.deb - -1. To ensure that `wkhtmltopdf` functions properly, copy the binaries to a location in your executable path and give them the necessary permission for execution: - - sudo cp /usr/local/bin/wkhtmlto* /usr/bin/ \ - && sudo chmod a+x /usr/bin/wk* - -### Set Up Virtualenv - -It's considered a best practice to isolate Odoo's Python modules from the modules included as part of the operating system to prevent unforeseen conflicts in the long run, especially after periodic OS updates. For that reason using `virtualenv` is highly recommended. - -1. Create a new `virtualenv` environment for Odoo 13 application: - - python3 -m venv /home//odoo-env - -2. Activate the `odoo-env` virtual environment you created in the previous step: - - source /home//odoo-env/bin/activate - -3. Update `pip3` using the following command: - - pip3 install --upgrade pip - -4. Install Python's wheel in the virtual environment: - - pip3 install wheel - -Let's review the virtual environment creation: - -* `python3 -m venv`: Runs `venv` module using Python 3, this module is in charge of creating the virtual environment. -* `/home//odoo-env`: Indicates the path used for the virtual Python environment. For the purpose of this guide, `home` directory of the current user was used but you can change it to any location that suits your needs as long as you remember to grant the `odoo` user with proper permissions afterward. - -### Install Odoo's Python modules - -1. Install the dependencies required by Odoo in the Python 3 environment: - - pip3 install -r /opt/odoo/doc/requirements.txt - pip3 install -r /opt/odoo/requirements.txt - -1. Check that all requirements are properly installed in your virtual environment: - - pip3 list - -1. Exit from the Python virtual environment by issuing the command: - - deactivate - -### Configure the Odoo Server - -1. Copy the included configuration file to `/etc/` and change its name to `odoo-server.conf` - - sudo cp /opt/odoo/debian/odoo.conf /etc/odoo-server.conf - -2. Modify the configuration file. The complete file should look similar to the following, depending on your deployment needs: - - {{< file "/etc/odoo-server.conf" conf >}} -[options] -admin_passwd = admin -db_host = postgresql.yourdomain.com -db_port = False -db_user = odoo -db_password = odoo_password -addons_path = /opt/odoo/addons -xmlrpc_port = 8069 -{{< /file >}} - -* `admin_passwd`: The password that allows administrative operations within Odoo GUI. Be sure to change `admin` to something more secure. -* `db_host`: The **postgresql** FQDN. -* `db_port`: Odoo uses PostgreSQL's default port `5432`, change this only if you're using custom PostgreSQL settings. -* `db_user`: Name of the PostgreSQL database user. -* `db_password`: Use the PostgreSQL `odoo` user password you created previously. -* `addons_path`: Default addons path. You can add custom paths separating them with commas: `` -* `xmlrpc_port`: Port that Odoo listens on. - -### Create an Odoo Service - -Create a systemd unit called `odoo-server` to allow your application to behave as a service. Create a new file at `/lib/systemd/system/odoo-server.service` and add the following, replace `/home/` with the directory where you setup your virtual Python environment: - -{{< file "/lib/systemd/system/odoo-server.service" shell >}} -[Unit] -Description=Odoo Open Source ERP and CRM - -[Service] -Type=simple -PermissionsStartOnly=true -SyslogIdentifier=odoo-server -User=odoo -Group=odoo -ExecStart=/home//odoo-env/bin/python3 /opt/odoo/odoo-bin --config=/etc/odoo-server.conf --addons-path=/opt/odoo/addons/ -WorkingDirectory=/opt/odoo/ -StandardOutput=journal+console - -[Install] -WantedBy=multi-user.target -{{< /file >}} - -### Change File Ownership and Permissions - -1. Change the `odoo-server` service permissions and ownership so only root can write to it, while the `odoo` user can only read and execute on it: - - sudo chmod 755 /lib/systemd/system/odoo-server.service \ - && sudo chown root: /lib/systemd/system/odoo-server.service - -2. Since the `odoo` user runs the application, change its ownership accordingly. Replace `/home/` with the directory where you setup your virtual Python environment: - - sudo chown -R odoo: /opt/odoo/ && sudo chown -R odoo: /home//odoo-env - -3. Protect the server configuration file. Change its ownership and permissions so no other non-root user can access it: - - sudo chown odoo: /etc/odoo-server.conf \ - && sudo chmod 640 /etc/odoo-server.conf - -### Test your Odoo Stack - -Confirm that everything is working as expected. - -1. Start the Odoo server: - - sudo systemctl start odoo-server - -1. Confirm that `odoo-server` is running: - - sudo systemctl status odoo-server - -1. In a browser, navigate to `odoo.yourdomain.com:8069` or `http://:8069`. If your proxy and your DNS configuration are working properly you are presented with Odoo's database creation screen: - - ![Odoo 13 Database Screen](odoo-13-first-screen.png "Odoo 13 Initial Screen") - -1. Fill in all the fields, check the **Demo data** box to populate your database with sample data, and then click on **Create database** button. - - ![Odoo 13 Create Database](odoo-13-create-db.png "Odoo 13 Create Database") - -1. In the browser, you should see a list of available apps, indicating that database creation was successful: - - ![Odoo 13 Welcome Screen](odoo-13-welcome.png "Odoo 13 Welcome Screen") - - The first time you create a database, Odoo may take several minutes to load all of its add-ons. Do not reload the page during this process. - -### Enable the Odoo Service - -1. Enable the `odoo-server` service to start automatically on reboot: - - sudo systemctl enable odoo-server - -1. Reboot your Linode from the Linode Manager. - -1. Check the Odoo logs to verify that the Odoo server is running without errors: - - sudo journalctl -u odoo-server - -## Back Up Odoo Databases - -If all components of the Odoo stack are running on a single server, it is simple to back up your databases using the Odoo web interface. However, this does not work with the configuration in this guide, since PostgreSQL was not installed on the **Odoo** Linode server. - -You have two options to backup your production database: - -1. You can install PostgreSQL 10 on the **Odoo** server using the procedure described on this guide. This installs `pg_dump` and other utilities, allowing you to use the Odoo GUI as before. Since Odoo configuration is explicit about database connection you do not have to worry about anything else. This method restores the database to the **PostgreSQL** server rather than **Odoo**. - -2. You can also use a procedure similar to the one described in our guide [How to Back Up Your PostgreSQL Database](/cloud/guides/back-up-a-postgresql-database) from the backend **PostgreSQL** server. - -### Update Odoo Modules - -Once you have backed up your production database you can update Odoo modules. - -From your **Odoo** server restart the Odoo service using the following flags to instruct the system to search for updates and apply any changes to modules: - - sudo service odoo-server restart -u all -d -{{< note >}} -From Odoo version 12 forward it is suggested that you update modules using Odoo's web interface whatever possible. -{{< /note >}} - -### Update your System - -If all your tests pass, you can safely update your installation. - -1. From your Linode, download the new code from source: - - cd /opt/odoo \ - && sudo git fetch origin 13.0 - -2. Apply the changes to your repository: - - sudo git reset --hard origin/13.0 -{{< note >}} -Do not confuse the Odoo system update with an Odoo **version** upgrade. With the method explained above, you are updating your Odoo application within the same version rather than **upgrading** to a newer Odoo version. Migrating from one version to another often requires several tests and manual modifications on the PostgreSQL database which are highly dependent on the version of Odoo you are upgrading from. -{{< /note >}} diff --git a/docs/guides/websites/wikis/install-mediawiki-on-ubuntu-1804/index.md b/docs/guides/websites/wikis/install-mediawiki-on-ubuntu-1804/index.md deleted file mode 100644 index f001b141ac3..00000000000 --- a/docs/guides/websites/wikis/install-mediawiki-on-ubuntu-1804/index.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -slug: install-mediawiki-on-ubuntu-1804 -title: Install MediaWiki on Ubuntu 18.04 -description: 'This guide will show you how to get started with the popular MediaWiki engine for powering wiki websites of all types and sizes on Ubuntu 18.04.' -authors: ["Linode"] -contributors: ["Linode"] -published: 2019-08-28 -keywords: ["mediawiki", "wiki", "web-applications"] -tags: ["wiki","ubuntu"] -license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' -aliases: [] -external_resources: - - '[MediaWiki Wiki](http://www.mediawiki.org/wiki/MediaWiki)' - - '[What is Media Wiki](https://www.mediawiki.org/wiki/Manual:What_is_MediaWiki%3F)' - - '[Media Wiki Extensions Blog](https://phabricator.wikimedia.org/source/extensions/browse/)' -relations: - platform: - key: install-mediawiki - keywords: - - distribution: Ubuntu 18.04 ---- - -MediaWiki is a popular, free wiki software package. It's the same software Wikipedia uses. It is fully dynamic and runs on a LAMP stack, taking advantage of the PHP language and the MySQL database backend. With easy installation and configuration, MediaWiki is a good solution when you need a familiar, full-featured, dynamic wiki engine. - -This guide assumes that you already have a working [LAMP stack](/cloud/guides/how-to-install-a-lamp-stack-on-ubuntu-18-04) running on Ubuntu. Your web accessible `DocumentRoot` should be located in `/var/www/html/example.com/public_html/`. You should be connected to your server via SSH and logged in as root. - -## Download and Unpack MediaWiki - -1. Change your working directory to Apache's `DocumentRoot` and download the latest release of MediaWiki. As of this writing, the latest stable release of MediaWiki is version 1.33.0. - - cd /var/www/html/example.com/ - sudo curl -O https://releases.wikimedia.org/mediawiki/1.33/mediawiki-1.33.0.tar.gz - - You will want to check for the latest version of this software regularly and upgrade to avoid allowing your site to become vulnerable to known security bugs. You can find the download location for the latest release by visiting the [MediaWiki homepage](http://www.mediawiki.org/wiki/MediaWiki). - -2. Decompress the package: - - sudo tar -xvf mediawiki-1.33.0.tar.gz - -3. Move the uncompressed `mediawiki-1.33.0` directory into your site's `public_html/` folder, renaming the directory to `mediawiki/` in the process. - - sudo mv mediawiki-1.33.0/ public_html/mediawiki/ - - The name of the directory beneath the `public_html/` will determine the path to your wiki. In this case, the wiki would be located at `example.com/mediawiki/`. You can copy the wiki to any publicly accessible location in the `public_html/` hierarchy. - -### Configure MySQL - -Mediawiki needs to communicate with a database to store information. Create a database and a user with a secure password, then grant all privileges on the new database to the user. - -1. Log in using the MySQL root password: - - sudo mysql -u root -p - -1. Create a database and a user with permissions for it. In this example, the database is called `my_wiki`, the user `media_wiki`, and password `password`. Be sure to enter your own password. This should be different from the root password for MySQL: - - CREATE DATABASE my_wiki; - CREATE USER 'media_wiki'@'localhost' IDENTIFIED BY 'password'; - GRANT ALL ON my_wiki.* TO 'media_wiki'@'localhost' IDENTIFIED BY 'password'; - - -## Configure MediaWiki - -Point your browser to the URL of your wiki, for example: `example.com/mediawiki/` and click the "Please set up the wiki first" link. The setup page contains everything you need to complete the installation. - -From the database section above, you will need: -- The database name -- DB username -- DB user's password - -Giving MediaWiki superuser access to your MySQL database allows it to create new accounts. If you plan on having a large number of users or content, consider setting up a second Linode as a [dedicated database server](/cloud/guides/standalone-mysql-server). - - After the installation is finished, MediaWiki will create a `LocalSettings.php` file, with the configurations from the installation process. Move the `LocalSettings.php` file to `/var/www/html/example.com/public_html/mediawiki/` and restrict access to the file: - - sudo chmod 700 /var/www/html/example.com/public_html/media/wiki/LocalSettings.php - -MediaWiki is now successfully installed and configured! - - -## Upgrade MediaWiki - -You can monitor the [MediaWiki development mailing list](https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce) to ensure that you are aware of all updates to the software. When upstream sources offer new releases, repeat the instructions for installing the MediaWiki software as needed.