# ercan.cloud

> Cloud, Kubernetes, AWS, infrastructure, and platform engineering writing by Ercan Ermis. Senior-engineer voice for a 201-level audience.

## Posts (full content)


---

# Local DynamoDB Grew Up: A Hands-On Look at ExtendDB

URL: https://ercan.cloud/local-dynamodb-grew-up-a-hands-on-look-at-extenddb/
Date: 2026-05-21T17:42:29+03:00


DynamoDB Local has been the laptop stand-in for AWS DynamoDB since 2013. It's a Java JAR, runs in memory or against a SQLite file, accepts almost any request shape, has no real auth, and treats Streams pretty loosely. It's fine for unit tests. It starts to crack the moment your code does anything beyond PutItem and GetItem.

ExtendDB v0.1.0 just shipped. It's a clean-room implementation of the DynamoDB wire protocol, written in Rust by AWS engineers, backed by PostgreSQL, Apache 2.0. The pitch is "DynamoDB Local, but you can take it seriously." This post is a hands-on look at whether that holds up, run as a side-by-side lab on a single Mac.

What ExtendDB actually is

ExtendDB isn't a fork of DynamoDB. There's no DynamoDB source code in it. What it does is speak the DynamoDB wire protocol (the JSON-1.0 X-Amz-Target dispatch that the AWS SDKs use), so an existing boto3 or AWS CLI client can be pointed at an ExtendDB endpoint with one flag change and it works, no modifications needed.

Under the hood it's a small, focused Rust workspace:

flowchart LR
    bin[extenddb<br/>CLI + daemon] --> server[extenddb-server<br/>HTTP + console]
    server --> engine[extenddb-engine<br/>DynamoDB op handlers]
    server --> auth[extenddb-auth<br/>SigV4 + IAM policy]
    engine --> core[extenddb-core<br/>types, expressions]
    engine --> storage[extenddb-storage<br/>trait definitions]
    storage --> pg[extenddb-storage-postgres<br/>PostgreSQL backend]

The extenddb-storage crate is just trait definitions. extenddb-storage-postgres is the only implementation today. The seam is clean enough that a different backend (sqlite, foundationdb, whatever) would mostly come down to implementing the traits. Here's the lifecycle trait, verbatim from crates/storage/src/lib.rs:

/// All methods receive `account_id` to scope operations to a single account.
/// This enables multi-account isolation: different accounts can have tables
/// with the same name without conflict.
pub trait TableEngine: Send + Sync {
    fn create_table(
        &self,
        account_id: &str,
        input: CreateTableInput,
    ) -> BoxFuture<'_, Result<TableDescription, StorageError>>;
    fn delete_table(
        &self,
        account_id: &str,
        input: DeleteTableInput,
    ) -> BoxFuture<'_, Result<TableDescription, StorageError>>;
    fn describe_table(
        &self,
        account_id: &str,
        input: DescribeTableInput,
    ) -> BoxFuture<'_, Result<TableDescription, StorageError>>;
    // ...list_tables, update_table, table_key_info
}

Two things to notice. First, every method takes an account_id. Account scoping isn't a layer above storage, it's threaded all the way through. Second, the trait uses BoxFuture for object safety. The module-level docs mention "RPITIT", but the actual methods landed as boxed futures, probably to keep the trait object-safe for the engine layer.

Lab setup

Both backends run on the same Mac. Postgres 18.4 from Homebrew, started with brew services start postgresql@18. ExtendDB built with cargo build --release, then:

$ ./target/release/extenddb init --config extenddb.toml
…creating catalog database extenddb_catalog…
…generating self-signed TLS certificate at ~/.extenddb/tls/cert.pem…
  Admin credentials (shown once, save them now)
  │  Username: admin
  │  Password: <redacted>

TLS is mandatory. The init command generates a 587-byte self-signed EC P-256 cert (openssl x509 -in cert.pem -noout -subject shows CN=extenddb self-signed, O=extenddb). Clients trust it via AWS_CA_BUNDLE=~/.extenddb/tls/cert.pem.

ExtendDB then runs with extenddb serve on https://127.0.0.1:8000. Test credentials get provisioned through the management API via the bundled devtools/provision-test-credentials script, which creates a test account, IAM user, access key, and an allow-all-dynamodb policy.

DynamoDB Local goes up next to it in Docker:

$ docker run -d --name ddb-local -p 8001:8000 \
    amazon/dynamodb-local:latest \
    -jar DynamoDBLocal.jar -inMemory -port 8000

A boto3 conftest fixture flips between the two endpoints based on the LAB_BACKEND env var, so every probe runs against either backend without code changes. Transcripts of every probe were captured as JSON lines. Everything in this post is quoted from those.

Baseline parity: CRUD walks fine

First thing to know is that the easy stuff just works. A PutItem with the full type zoo:

ddb.put_item(TableName="lab_alpha", Item={
    "pk":         {"S": "user#1"},
    "sk":         {"S": "profile"},
    "name":       {"S": "Alice"},
    "age":        {"N": "30"},
    "active":     {"BOOL": True},
    "tags":       {"L": [{"S": "rust"}, {"S": "postgres"}]},
    "meta":       {"M": {"city": {"S": "Istanbul"}}},
    "deleted_at": {"NULL": True},
    "blob":       {"B": b"\x00\x01\x02"},
})

Same call, both backends, same response shape. Ten probes covering CreateTable (with a GSI), strong-consistent GetItem, Query on partition + sort range, Query on the GSI, Scan with a filter, UpdateItem with combined SET/REMOVE/ADD plus a ConditionExpression, DeleteItem with a condition, BatchWriteItem of 25 items, and a TransactWriteItems with an intentional condition-check conflict. All pass against both backends.

One small exception worth noting. ExtendDB's BatchWriteItem response omits the UnprocessedItems key entirely when the batch fully succeeded. Real DynamoDB and DynamoDB Local both return "UnprocessedItems": {}. It's the kind of drift that boto3 papers over with resp.get("UnprocessedItems", {}), but a strict client could trip on it.

So at the level of "boto3 code that ran against DynamoDB yesterday runs against ExtendDB today", the answer is yes. The interesting differences start where DynamoDB Local has historically been waved at.

Where they diverge

Auth is real, sort of

The popular shorthand "DynamoDB Local has no auth" turns out to be slightly wrong. Both backends reject a completely unsigned request:

# B1: no Authorization header on either
extenddb         -> 400 com.amazon.coral.service#MissingAuthenticationToken
dynamodb-local   -> 400 com.amazonaws.dynamodb.v20120810#MissingAuthenticationToken

The real gap shows up in B2. Send any Authorization header that parses as SigV4 (a made-up access key id, a Signature=deadbeef, anything), and:

# B2: bogus but parseable Authorization header
extenddb         -> 400 UnrecognizedClientException
                       "The security token included in the request is invalid."
dynamodb-local   -> 200 {"TableNames":[]}

ExtendDB looks up the access key against its IAM store and rejects when it doesn't exist. DynamoDB Local doesn't look at the access key at all. It accepts the request the moment the header parses. That's the gap.

Small but useful detail: ExtendDB access keys are prefixed AKIAEXTENDDB (or ASIAEXTENDDB for sessions), so they can't be confused with real AWS keys in logs or grep output.

Streams shaped like the real thing

Both backends implement Streams. The shape is different. Enable a stream with NEW_AND_OLD_IMAGES, write an item, overwrite it, delete it, then drain the stream:

# Naive single-shard read against ExtendDB returns zero records.
# ExtendDB partitions stream records across 4 shards by partition-key hash.
records = []
for shard in describe_stream(stream_arn)["StreamDescription"]["Shards"]:
    it = get_shard_iterator(StreamArn=stream_arn,
                            ShardId=shard["ShardId"],
                            ShardIteratorType="TRIM_HORIZON")["ShardIterator"]
    # ...drain pages until empty

That matches how real DynamoDB shards streams. DynamoDB Local takes a simpler path and puts everything on a single shard, which is why naive consumers happen to work against it and then surprise their authors in production.

One v0.1.0 gotcha worth surfacing. Deleting a stream-enabled table and recreating it with the same name during the same server lifetime fails with duplicate key value violates unique constraint "stream_shards_pkey". The shard rows from the old table don't get fully cleared. Workaround in the lab harness is to suffix table names with a timestamp.

TTL with real REMOVE records

ExtendDB runs a TTL sweeper. Drop in an already-expired item, wait, and the item is gone:

ddb.put_item(TableName="lab_ttl_...",
             Item={"pk": {"S": "doomed"},
                   "expires_at": {"N": str(int(time.time()) - 60)},
                   "payload": {"S": "x"}})
# 60 seconds later
ddb.get_item(...)  # no Item

More importantly, the deletion emits a stream record with the spec'd service identity:

INSERT  userIdentity = None
REMOVE  userIdentity = {'PrincipalId': 'dynamodb.amazonaws.com', 'Type': 'Service'}

That userIdentity matters. It's what real consumer code uses to tell a user-initiated DeleteItem apart from background TTL expiry. Two small drifts here. First, ExtendDB returns the keys as Type / PrincipalId (PascalCase), where the DynamoDB spec uses type / principalId. Code that does record["userIdentity"]["type"] will need a fallback. Second, the docs reference a ttl_deletion_target_seconds runtime setting; in v0.1.0 it isn't an exposed key, and the sweep interval is hardcoded at 60 s in crates/storage-postgres/src/ttl_worker.rs. DynamoDB Local skips this whole story: no sweeper at all, expired items stay in the table forever.

Multi-account isolation that works

Provisioned a second account lab_b, created a user, access key, and an allow-all-dynamodb policy. Then from two boto3 clients (one signed as each account), both created a table with the same name lab_shared_name and put a marker row each:

Account A reads back  -> {"owner": "from-a", "pk": "x"}
Account B reads back  -> {"owner": "from-b", "pk": "x"}

Same table name, different account, separate rows, no cross-talk. DynamoDB Local has exactly one namespace: same name, same table.

You can look inside

ExtendDB stores items in plain PostgreSQL. There's one shared extenddb data database (account isolation is enforced at the row level via account_id, not by separate databases). Each DynamoDB table maps to a Postgres table named _ddb_<uuid>. A row is pk | item_data (JSONB):

$ psql -d extenddb -c "SELECT * FROM _ddb_f6453e99-3173-... LIMIT 2;"
   pk    |                                  item_data
---------+------------------------------------------------------------------------------
 item-001| {"pk": {"S": "item-001"}, "owner": {"S": "account-a"}, "value": {"N": "42"}}
 item-002| {"pk": {"S": "item-002"}, "owner": {"S": "account-a"}, "value": {"N": "99"}}

For backup, restore, replication, debugging, or just satisfying curiosity, this is a different world from DynamoDB Local's opaque in-memory or SQLite file.

Crash safety and HTTPS

kill -9 the ExtendDB process mid-write, restart, read the canary back:

== Read canary after restart ==
"Item": { "payload": {"S": "survives-crash"}, "pk": {"S": "canary"} }

Postgres durability rather than process memory. DynamoDB Local in -inMemory mode loses everything on restart by design.

HTTPS is mandatory but it isn't implemented as outright refusal. A plain-HTTP request to ExtendDB's port returns 301 (redirect to HTTPS). No application data gets served over plain HTTP, but the connection is accepted long enough to send the redirect. DynamoDB Local serves HTTP normally.

Two design choices worth calling out

The first one is how streams stay consistent. From crates/storage/src/lib.rs:

/// Parameters for capturing a stream record within a data write transaction.
///
/// When present, the storage backend inserts the stream record in the same
/// transaction as the data write, guaranteeing atomicity.
pub struct StreamCapture {
    pub view_type: StreamViewType,
    pub user_identity: Option<UserIdentity>,
    pub region: Arc<str>,
}

The stream record write lives inside the same Postgres transaction as the item write. No "data committed but stream missed" or "stream wrote but data rolled back" possible. Both happen, or neither does. It's the kind of thing real DynamoDB hides behind its proprietary storage, and it's satisfying to see the seam explicitly in the trait.

The second is a small, surprising defensive choice. From docs/differences-from-dynamodb.md:

| TTL attribute name | Any UTF-8 string (1 to 255 bytes) | Restricted to [a-zA-Z0-9._-]+ (1 to 255 bytes). Names with spaces, quotes, or other special characters are rejected. This eliminates SQL injection risk in the TTL expression index. |

DynamoDB allows any UTF-8 in the TTL attribute name. ExtendDB rejects anything outside [a-zA-Z0-9._-]+. The reason is that the TTL attribute name is interpolated into a Postgres index expression, and a permissive name set would force quoting that could be exploited. It's a clean-room implementation making a security trade-off the original probably never had to think about.

When NOT to use it

A short list of concrete gaps, mixing genuinely missing features with v0.1.0 drift:

Not implemented at all: Global Tables, DAX, PartiQL (ExecuteStatement / BatchExecuteStatement), federated SAML/OIDC auth, Kinesis streaming destinations.

Different shape: ImportTable / ExportTableToPointInTime go to a local filesystem path instead of an S3 bucket. On-demand capacity has a fixed initial burst and no autoscaling.

Wire-protocol drift to watch out for: BatchWriteItem omits UnprocessedItems: {} from successful responses. userIdentity fields use PascalCase keys (Type, PrincipalId) rather than the DynamoDB-spec camelCase. Stream consumers must drain all shards (real DynamoDB behavior, but DynamoDB Local users may have only ever read shard 0).

v0.1.0 bug: a stream-enabled table can't be deleted and recreated with the same name within the same server lifetime (stream_shards_pkey collision). Rotate table names in tests.

Docs drift: the ttl_deletion_target_seconds runtime setting referenced in the differences doc isn't currently exposed; sweep interval is hardcoded.

None of these are reasons to avoid ExtendDB if you're hitting DynamoDB Local's limitations. They're reasons to test your code against ExtendDB before you assume the wire is identical.

Verdict

If you've ever written test code that papered over DynamoDB Local (mocked auth out, skipped Streams assertions, stubbed TTL) and quietly worried the real service would behave differently, ExtendDB is interesting. It's the first local DynamoDB stand-in that takes auth, Streams, TTL, multi-account isolation, and durability seriously, and the storage layer is plain Postgres you can psql into.

For CI pipelines that need realistic auth and stream semantics, on-prem or air-gapped deployments, and dev teams that have been losing time to "well, it works against DynamoDB Local...", v0.1.0 is already a useful tool. For anything that depends on byte-exact wire compatibility with the real service, do your own pass first.

Links:

Repo: https://github.com/ExtendDB/extenddb

Release: https://github.com/ExtendDB/extenddb/releases/tag/v0.1.0

Getting started: https://github.com/ExtendDB/extenddb/blob/v0.1.0/docs/getting-started.md

Behavioral differences: https://github.com/ExtendDB/extenddb/blob/v0.1.0/docs/differences-from-dynamodb.md

DynamoDB Local Docker image: https://hub.docker.com/r/amazon/dynamodb-local

AWS DynamoDB Local docs: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html

DynamoDB TTL stream userIdentity spec: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-streams.html

---

# How I Found a Hidden Cloudflare Bug on a Sunday mid-night (The joy of curl)

URL: https://ercan.cloud/how-i-found-a-hidden-cloudflare-bug-on-a-sunday-mid-night/
Date: 2026-05-10T00:44:28+03:00

It was supposed to be a quick weekend project. You know the kind: "I'll just spin up an egress VM, route some traffic through it, sip my ...


It was supposed to be a quick weekend project. You know the kind: "I'll just spin up an egress VM, route some traffic through it, sip my coffee, and be done by lunch." Reader, I was not done by lunch.

The Setup

I was building a small access tier for some dev environments: an egress VM with Cloudflare Zero Trust in front of it, hostname-based routing for two specific dev domains, and AWS WAF on the other side checking the source IP. Pretty standard "give my team secure access without exposing things to the public internet" stuff.

Everything went smoothly until I hit the part where you need to enable TLS decryption. This is the magic that lets Cloudflare's gateway actually inspect traffic and route it based on hostname rather than just IP. Without it, your beautiful hostname rules sit there looking pretty and doing absolutely nothing.

The Tutorial Trail

I followed the official Cloudflare learning path. I followed three different blog posts. I followed the YouTube tutorial from someone whose voice sounded like they really wanted me to subscribe. They all said the same thing:

Go to Zero Trust > Settings > Network > Firewall > TLS decryption and flip the switch.

So I went there. The switch flipped. Then a red banner appeared at the top of the screen that simply said:

Error configuring TLS setting.

That was it. No code. No hint. No "click here for details" link. Just a vibe of "something went wrong, good luck."

I tried again. Same error. I refreshed. Same error. I tried in Firefox. Same error. I cleared the cache. Same error. I considered making coffee. Made coffee. Same error.

The Detective Work

Here is where my favorite tool comes in. When the UI refuses to tell you what is wrong, the network tab and curl will. I opened developer tools, watched the request that the toggle was sending, copied it as curl, and ran it locally:

{
  "result": null,
  "success": false,
  "errors": &#91;
    {
      "code": 2211,
      "message": "TLS decryption cannot be enabled without a certificate. Please disable TLS encryption or configure a certificate using the 'certificate' setting."
    }
  ],
  "messages": &#91;]
}

There it was. A clear, helpful, actionable error message. The kind of message that, if shown in the UI, would have saved me about ninety minutes of my life. Instead it was hiding inside the JSON response, swallowed somewhere between the API and the screen.

The message basically said: "you need a certificate first, you absolute potato." (I am paraphrasing.)

The Real Fix

Here is the thing nobody on the internet seems to mention: the certificate management has moved.

Every guide I read pointed to the old location. The actual current location is buried in a different section entirely. Specifically:

Zero Trust > Traffic policies > Traffic settings > Certificates (tab at the top)

Not Network. Not Firewall. Traffic policies > Traffic settings.

You generate a Cloudflare-managed certificate there. Then you mark it as Available and In-Use. Then you wait five to ten minutes (this part also nobody mentions, you just watch a status field and quietly question your life choices). Once the certificate is fully active, you can finally go back and enable TLS decryption.

It just works. The toggle flips. The red banner stays away. The world makes sense again.

The Moral

Two things stuck with me from this little adventure.

First: the docs were not wrong, they were just out of date. Cloudflare moves fast and reorganizes their dashboard often. This is not a complaint, just a reality. When following any guide for any rapidly evolving product, expect that the menu paths might have shifted. The concepts are the same, the buttons are not always where the screenshots say they are.

Second: the network tab is your best friend. When a UI gives you a useless error, the API almost always knows more. Browser dev tools, curl, or even just reading the raw response is the difference between debugging in five minutes and debugging in five hours. UI error handling is a thin layer painted over a much more verbose system, and that system is usually right there waiting for you to ask it nicely.

The Bonus

I did report this to Cloudflare. The error code 2211 has a perfectly good message attached to it. It just is not making it to the screen. Hopefully someone catches it and a future weekend warrior gets a helpful banner instead of a vague one.

Until then, if you ever see "Error configuring TLS setting" with no further detail, the answer is probably:

Generate a certificate first (under Traffic policies > Traffic settings)

Mark it Available and In-Use

Wait

Then enable TLS decryption

Now if you will excuse me, I have an egress VM that finally works, and a coffee that has gone cold twice.

Cheers from the Sunday mid-night (02:42 AM) debugging desk.

---

# Why Your SSH Is Yelling About Quantum Computers (And How to Fix It)

URL: https://ercan.cloud/why-your-ssh-is-yelling-about-quantum-computers-and-how-to-fix-it/
Date: 2026-04-17T13:12:28+03:00


You SSH into your server and see this:

** WARNING: connection is not using a post-quantum key exchange algorithm.
** This session may be vulnerable to "store now, decrypt later" attacks.

Scary. Let's break it down.

The Threat: Store Now, Decrypt Later

Today's SSH encryption is mathematically strong; breaking it would take a classical computer millions of years.

But quantum computers play by different rules. A sufficiently powerful quantum computer running Shor's algorithm can break the math that protects most of today's public-key cryptography in hours, not millions of years.

Here's the unsettling part: quantum computers powerful enough to do this don't exist yet. So why worry now?

Because adversaries, nation-states especially, are already recording encrypted traffic today with the plan to decrypt it later once quantum hardware matures. This is the "store now, decrypt later" (SNDL) attack. If you're sending anything sensitive over SSH that needs to stay secret for the next 10–20 years, you're already at risk.

This is not science fiction. The NSA, CISA, and NIST have all issued guidance on migrating to post-quantum cryptography now.

What Is Key Exchange, Anyway?

Before SSH can encrypt your session, both sides need to agree on a shared secret, a temporary key used to encrypt everything else. This negotiation is called key exchange (Kex).

Think of it like this: two people want to agree on a secret color without anyone watching being able to figure it out. Key exchange is the cryptographic version of that trick (look up Diffie-Hellman if you want the details, it's a beautiful idea).

The problem: the math behind classical key exchange (Diffie-Hellman, elliptic curves) is vulnerable to quantum computers.

Classical vs Post-Quantum Key Exchange

Here's a quick tour of what SSH supports and where each algorithm stands:

Classical (vulnerable to quantum computers)

AlgorithmNotesdiffie-hellman-group1-sha1Avoid. Weak, deprecated, ancient.diffie-hellman-group14-sha1Avoid. SHA-1 is broken.diffie-hellman-group14-sha256Acceptable today, not future-proof.diffie-hellman-group16-sha512Stronger DH, still classically vulnerable.diffie-hellman-group18-sha512Strongest DH group, still classically vulnerable.diffie-hellman-group-exchange-sha256Negotiates key size, common default.ecdh-sha2-nistp256Elliptic curve, fast. NIST curve (some trust concerns).ecdh-sha2-nistp384Same, larger key.ecdh-sha2-nistp521Same, largest key.curve25519-sha256Best classical option. Modern, trusted, fast.curve25519-sha256@libssh.orgSame algorithm, older name.
All of the above are broken by a quantum computer running Shor's algorithm.

Post-Quantum (safe against quantum computers)

AlgorithmOpenSSH VersionNotessntrup761x25519-sha512@openssh.com8.5+Available now. Hybrid: NTRU Prime + X25519.mlkem768x25519-sha2569.9+Gold standard. Hybrid: ML-KEM-768 + X25519. NIST FIPS 203 standardized.
Both are hybrid algorithms they layer a post-quantum algorithm on top of a classical one. This means:

If the PQ algorithm has a flaw → X25519 still protects you (classical security).

If a quantum computer attacks → the PQ layer still protects you.

You get the best of both worlds. Hybrids are the recommended approach during this transition period.

mlkem768x25519-sha256 is the one to prefer long-term. ML-KEM (formerly Kyber) was selected by NIST as the standard post-quantum key encapsulation mechanism in 2024. It's in OpenSSH 9.9+.

sntrup761x25519-sha512@openssh.com uses NTRU Prime, an older PQ candidate. Still solid, available in older OpenSSH. Use it if you're on 8.5–9.8.

Why the Warning Appears

OpenSSH 8.5+ enables PQ by default in its built-in KexAlgorithms list. But if your sshd_config has a hardcoded KexAlgorithms line, it overrides the defaults and, whoever wrote that line may not have included PQ algorithms.

That's exactly what triggers the warning: the server is advertising only classical algorithms, so the client (which prefers PQ) falls back to classical.

Check your server:
grep -i kexalgorithms /etc/ssh/sshd_config

If you see a line with only classical algorithms (like curve25519-sha256,diffie-hellman-group-exchange-sha256), that's your culprit.

The Fix

Step 1: Edit /etc/ssh/sshd_config

Add the PQ algorithm at the front of the list. OpenSSH negotiates in order; first match wins.

If you're on OpenSSH 9.9+ (check with ssh -V):

KexAlgorithms mlkem768x25519-sha256,sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

If you're on OpenSSH 8.5–9.8 (like Ubuntu 24.04's bundled 9.6p1):

KexAlgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

Keep the classical algorithms as fallbacks. Some clients (embedded systems, old hardware) don't support PQ yet and need them to connect.

To see every algorithm your OpenSSH build actually supports: ssh -Q kex

Step 2: Validate and Restart

Always validate before restarting, a typo in sshd_config can lock you out: sudo sshd -t

If no errors: sudo systemctl restart sshd

Step 3: Verify It Worked

From a client machine, connect with verbose output and grep for the negotiated kex: ssh -v user@yourserver 2>&1 | grep "kex_"

Look for something like: debug1: kex: algorithm: sntrup761x25519-sha512@openssh.com or from the server side, check an active connection: sudo ss -tnp | grep :22

What About the Client Side?

The same fix applies to your SSH client config (~/.ssh/config or /etc/ssh/ssh_config). If your client has a hardcoded KexAlgorithms line without PQ, it'll fall back to classical even when connecting to a PQ-capable server.

To suppress the warning on the client side while you work on fixing servers:

# ~/.ssh/config
Host *
    KexAlgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,diffie-hellman-group-exchange-sha256

But the real fix is upgrading both ends.

TL;DR

QuestionAnswerAm I being hacked right now?No. This is about future quantum computers.Do I need to act immediately?Sensitive long-lived data: yes. Personal servers: soon.Which algorithm should I use?mlkem768x25519-sha256 (OpenSSH 9.9+) or sntrup761x25519-sha512@openssh.com (8.5+)Will it break old clients?No, keep classical algorithms as fallbacks.Is this the only thing to quantum-proof?No. Host keys and user auth keys (RSA, ECDSA) also need attention, that's a separate post.
The quantum era isn't here yet, but the window to prepare is now. Updating KexAlgorithms takes two minutes and is one of the easiest wins in modern server hardening.

Ubuntu 24.04 ships OpenSSH 9.6p1, which supports sntrup761x25519-sha512@openssh.com out of the box. OpenSSH 9.9+ (available via PPA or newer distros) adds the NIST-standardized mlkem768x25519-sha256.

---

# IaC-First: Why I am Never Touch the AWS Console in Production

URL: https://ercan.cloud/iac-first-why-we-never-touch-the-aws-console-in-production/
Date: 2026-04-16T19:02:42+03:00


"Never touch the AWS console in production" sounds like an extreme rule. It is not. It is the most important operational discipline in a cloud-native team, and the cost of violating it accumulates silently until it causes a major incident.

This post explains why, and how to enforce IaC-first development in a real team.

The State Drift Problem

Terraform (and OpenTofu) maintains a state file that represents what infrastructure exists. When you apply, Terraform compares the state file against your configuration and makes the minimum set of changes to bring reality in line with the configuration.

When you click around in the AWS console and create or modify resources, you are changing reality without changing the state file.

Now: The next plan will show drift: Terraform wants to destroy or recreate resources that "should not exist" (because they are not in the configuration).

Or worse: Terraform's state says the resource exists with config A, but the console changed it to config B. plan shows "no changes" even though the resource is misconfigured.

Or: someone adds a resource in the console, it works, gets depended upon by other resources, and then an apply runs and deletes it because it is not in the state.

The console is not the source of truth. Your IaC is. The console is a lie.

The Import Workflow: The Correct Response to "It Already Exists"

When a resource exists in AWS but not in your Terraform configuration (created by someone in the console, created manually via CLI, or migrated from another tool), the correct action is to import it:

Import an existing resource into Terraform state

./deploy.sh --infra --import aws_s3_bucket.service existing-bucket-name
./deploy.sh --infra --import aws_dynamodb_table.service_files serviceFiles
./deploy.sh --infra --import aws_lambda_function.service_api myapp-service-api

After importing, run plan:
./deploy.sh --infra

The plan shows the diff between what exists in AWS and what your configuration says. Fix the configuration until the plan shows no unexpected changes. Then apply to bring the state file in sync.

Never delete a resource from the console and re-create it via Terraform. This destroys data and breaks dependencies. Import → reconcile → apply.

The One Legitimate Console Use

The console is appropriate for:

Read-only exploration. CloudWatch logs, CloudWatch metrics, Lambda invocation history, DynamoDB item inspection.

Emergency operations with immediate import. If production is down and the fix requires a console change, make the change; but document it and import it into Terraform within the same working session.

That is it. Everything that creates, modifies, or deletes resources belongs in IaC.

The Deploy Script as the Single Entry Point

Raw terraform apply or tofu apply commands run directly on the host are dangerous because they bypass your team's standard deploy workflow (validation, lint, plan review). Create a deploy script that wraps all Terraform operations:

#!/bin/bash
# deploy.sh - the only way to interact with infrastructure

case "$1" in
    --infra)
        case "$2" in
            --apply)   run_apply ;;
            --plan)    run_plan ;;
            --validate) run_validate ;;
            --tflint)  run_tflint ;;
            --import)  run_import "$3" "$4" ;;
            *)         run_plan ;;  # default: plan only (safe)
        esac
        ;;
esac

Key properties of a good deploy script:

Default to plan. Running ./deploy.sh --infra with no second argument should show the plan, never apply. Applying requires explicit intent (--apply).

Always validate before apply. Run terraform validate and tflint before any apply. Reject the deploy if either fails.

Always show the plan before applying. Even for --apply, show the plan output and require confirmation in interactive mode.

Consistent var-file. Always include the same var-file (vars/production.tfvars). Never apply without it.

Plan Review as a Gate

Before any infrastructure PR merges, the plan output should be visible in the PR:
## Terraform Plan

Terraform will perform the following actions:
module.service_api.aws_lambda_function.service_api will be updated in-place
~ resource "aws_lambda_function" "service_api" {
~ timeout = 30 -> 60
}

Plan: 0 to add, 1 to change, 0 to destroy.

module.service_api.aws_lambda_function.service_api will be updated in-place

~ resource "aws_lambda_function" "service_api" {
~ timeout = 30 -> 60
}

Plan: 0 to add, 1 to change, 0 to destroy.

This accomplishes two things:

The reviewer can see exactly what will change in production before approving.

The plan output is a historical record of what was intended at merge time.

A plan that shows unexpected destroys (N to destroy) should block the PR until the author explains why.

The No-Exclude Rule

A common mistake when an apply fails: exclude the failing resource and apply anyway.

# WRONG tofu apply -exclude='aws_bedrockagent_knowledge_base.kb'

This creates a partial apply: some resources were updated, some were not. The state file now reflects a partially-applied configuration. The excluded resource is out of sync with everything that depends on it.

The correct response to a failing resource:

Let the apply finish. Do not interrupt it.

Understand why the resource failed.

Fix the configuration or import the existing resource.

Run apply again until zero errors.

If the error is "resource already exists": import it.
If the error is a config mismatch: fix the configuration to match the existing resource.
If the error is a dependency ordering problem: fix depends_on.

Never: exclude, comment out, or otherwise skip resources. The IaC configuration must always reflect reality.

The Forced

Terraform uses a lock table (DynamoDB) to prevent concurrent applies. If a lock is stuck (Lambda timed out mid-apply, process was killed), you will see:

Error: Error locking state: Error acquiring the state lock

The tempting fix: terraform force-unlock. This is dangerous if another apply is actually running - force-unlocking a live apply causes corruption.

The correct response:

Check if any apply is actually running (check CloudWatch for the deploy Lambda, check with teammates).

If no apply is running and you are certain the lock is stale, then force-unlock.

Document that you forced a lock - it is a signal to investigate why the previous apply did not clean up properly.

Never force-unlock as a first response. Always verify there is no live apply first.

Drift Detection

Even with strict IaC discipline, drift accumulates. AWS may auto-modify resources (e.g., Cognito updating a Lambda trigger association), or a team member may make an emergency console change and forget to import it.

Schedule a regular plan run (weekly or after significant activity periods) with no apply:

# In CI/CD: weekly scheduled plan ./deploy.sh --infra --plan 2>&1 | tee plan-output.txt # Alert if plan shows unexpected changes

If the plan shows changes that should not exist, investigate before the next apply destroys them.

Key Takeaways

The AWS console is for reading, not writing. Every infrastructure change belongs in IaC.

When a resource exists in AWS but not in Terraform: import it, reconcile the config, then apply.

Never use -exclude to work around a failing resource. Fix the root cause.

Wrap all Terraform operations in a deploy script. Default to plan; require explicit intent to apply.

Put the plan output in every infrastructure PR. Unexpected destroys should block the PR.

Never force-unlock without verifying no live apply is running.

Run scheduled plan checks to detect drift early.

---

# I dropped my Google Pixel 9 XL Pro from 6th floor balcony to the street

URL: https://ercan.cloud/i-dropped-my-google-pixel-9-xl-pro-from-6th-floor-balcony-to-the-street/
Date: 2026-02-01T02:14:25+03:00

My phone slipped out of my hand and executed a textbook, no-hesitation, full-commitment dive from the 6th floor straight to the street.


Gravity 1 – Google Pixel 9 Pro XL 0

Tonight I went to my friend’s place. Great conversation, good laughs, zero awareness of physics. By the time I got home, it was 02:00 AM, that dangerous hour where confidence is high and grip strength is low.

As usual, before bed, I stepped onto my lovely balcony to get some fresh air. Phone in hand.
First, I was replying to some messages. Normal life stuff. Very safe. Very controlled.

Then I looked up. The moon was amazing. One of those nights where you think, yeah, this deserves a photo. 

So I slightly adjusted my grip, moved the phone to camera mode, leaned a bit…

And at that exact moment, gravity remembered me.

I calmly said:

“Oh… I was always wondering what would happen if I dropped my phone from here.”

Curiosity answered immediately. At midnight. With concrete.

Google Pixel 9 XL Pro Is Stronger Than Gravity

While running downstairs in mild disbelief, I noticed something unreal:

My Sony WH-1000XM6 headphones were still connected via Bluetooth.

Sound was still playing. The phone was still alive. Like it just bounced and said, “You done?”

I picked it up.

Screen: alive

No dead pixels

Just scratches

Bezel cracked

The back glass shattered into modern art

The camera cover popped out dramatically

Camera? Works.
Flex cables? Still loyal.

This thing fell from the 6th floor, not from a desk or from a pocket. 

Samsung Era: From Android 1.5 to Galaxy S21 Ultra

Let’s be clear: I’m not new to Android.

I’ve been using Samsung Android phones since Android 1.5 (Cupcake) 2009, with the legendary Samsung GT-I7500.

Since then, I’ve lived through:

Android 1.5 Cupcake

2.1 Eclair

2.3 Gingerbread

4.x Ice Cream Sandwich & Jelly Bean

5.x Lollipop

6 Marshmallow

7 Nougat

8 Oreo

9 Pie

10

11

12

Up to Samsung Galaxy S21 Ultra

That’s more Android versions than some people have email addresses.

And honestly? If this phone had been a Samsung and fallen from the 6th floor, it would’ve:

Bootlooped

Forgotten its IMEI

Opened Safe Mode

Asked me why I did this

And emotionally disconnected from the motherboard

Respectfully.

Smartphones Are Not Magic (I Checked)

Of course it’s repairable. Smartphones aren’t complicated; they just like to act mysterious.

I’ve been fixing phones since I was 12. Back when I was replacing Nokia keypad LEDs from boring white to blue, green, and red, just because I could.

That’s where I learned:

Soldering

Patience

And how to stay calm while something absolutely does not want to work

So I looked at my shattered Pixel and thought:

“Yeah… AliExpress will fix this.”

AliExpress Saved My Night (€38)

I ordered:

Main frame

Back glass

Total cost: 38 EUR

Bonus twist:
My phone was white but now it’s going to be black.

Not a repair. It's a rebirth!

Case or No Case? Still No Case.

Would a case have helped? Probably.

Will I use one? No.

I like my phones like I like my decisions: Unprotected and confident.

And While We’re Here: Fuck the iPhone

Let’s be honest.

If this were an iPhone:

The screen would be dust

Face ID would forget my face

The repair would cost half a kidney

Apple would suggest buying a new one

And a Genius would look at me like I personally offended Steve Jobs

So yeah. Fuck the iPhone!

Finding the Good in Bad Situations

Life drops your phone from the 6th floor sometimes.

But if:

Your Bluetooth stays connected

Your screen survives

And your phone is still repairable

You’re doing fine.

I get:

A fixed phone

A new color

A fun repair project

And a good story

Win!

Final Thanks

Thanks to:

AliExpress, for enabling self-repair

Google engineers, for building a tank disguised as a phone

God, for giving me hands that can fix broken things 🙏🏻

And to gravity: We’ll meet again.

P.S. I’ll write another blog post after I finish repairing this thing.

Spoiler: It will live again. Stay tuned 😌📱

---

# AWS S3 New Feature: Re-encryption without Movement

URL: https://ercan.cloud/aws-s3-new-feature-re-encryption-without-movement/
Date: 2026-01-30T11:51:05+03:00


The recent release of the UpdateObjectEncryption API marks a significant shift in how we manage data security at scale. Historically, changing the encryption of an S3 object was a "physical" operation; you had to move the bits. Now, it’s a "logical" metadata operation.

Technical Deep Dive: Re-encryption without Movement

The "magic" behind this update lies in Envelope Encryption. In the legacy CopyObject In the workflow, S3 had to decrypt the actual data using the old key and re-encrypt it with the new key, effectively creating a new file.

With the UpdateObjectEncryption API, S3 does not touch the underlying data blocks. Instead, it only interacts with the Data Key (DK).

S3 retrieves the encrypted Data Key stored in the object's metadata.

It uses the old Master Key to decrypt that Data Key.

It immediately re-encrypts that same Data Key using the new KMS Master Key.

The object metadata is updated atomically with the new encrypted Data Key.

Because the data itself (the ciphertext) remains identical, the ETag (which is a hash of the data) and the Creation Date stay exactly the same.

Pros and Cons

Pros

Zero Latency on Data: Since no data is moved, the operation takes milliseconds regardless of the file size, from 1 KB to 5 TB.

Storage Integrity: It preserves the Last-Modified date, ETag, and Version ID. This is critical for applications that use ETags for cache validation.

Archive-Friendly: You can upgrade the encryption on Glacier objects without first "restoring" them. They stay in cold storage.

Cost Efficiency: You avoid the data retrieval fees (GET) and data transfer fees associated with copying. You only pay for the API call (PUT price).

Intelligent-Tiering Safety: Objects in S3 Intelligent-Tiering do not get "reset" to the Frequent Access tier, preserving your cost-optimization progress.

Cons

KMS Only: You can only update to SSE-KMS. It does not support SSE-C or DSSE-KMS.

No "Unencrypted" Support: The source object must already be encrypted (SSE-S3 or SSE-KMS). If it’s one of the rare legacy "Plaintext" objects, this API will fail.

Object Lock Barrier: If an object is under Legal Hold or Retention Mode, you cannot update its encryption until the lock is removed.

KMS ARN Requirement: You cannot use KMS Key Aliases; you must provide the full Amazon Resource Name (ARN).

Comparison: UpdateObjectEncryption vs. CopyObject

FeatureUpdateObjectEncryption (New)CopyObject (Legacy)Data MovementNone (Metadata update only)Full data copyObject MetadataPreserved (ETag, Creation Date)Reset (New ETag, New Date)Glacier SupportWorks while archivedRequires "Restore" firstThroughputHigh (Atomic/Instant)Limited by object size/bandwidthCostPUT Request only ($0.005/1k)GET + PUT + Data Retrieval feesObject LockBlockedCreates new version (if versioned)
Decision Guide: Which should you prefer?

1. Use UpdateObjectEncryption when:

Status: Compliance Audit. You need to rotate from AWS-managed keys (SSE-S3) to Customer-managed keys (SSE-KMS) to meet regulatory requirements.

Status: Archived Data. You have petabytes of data in Glacier Deep Archive that need an encryption upgrade without the massive cost of restoring them.

Status: Massive Objects. You have 5 TB objects where a copy operation would take hours and risk a timeout.

Status: ETag Dependent. Your application logic relies on the ETag remaining constant for sync/caching.

2. Use CopyObject when:

Status: Unencrypted Source. You are dealing with very old objects that have no encryption at all.

Status: Cross-Region/Account. You need to move the data to a different bucket or region while changing encryption.

Status: SSE-C Requirements. You need to use Customer-Provided keys (SSE-C), which the new API does not support.

Status: Locked Objects. You need to update encryption on an object that is permanently locked (Compliance Mode); in this case, a new copy/version is your only path.

Note on Scale: If you are dealing with millions of objects, do not call this API in a loop. Use S3 Batch Operations. AWS has integrated this new API directly into the Batch console, allowing you to run a "re-encryption" job across an entire bucket with a few clicks.

You can read the official AWS news via https://aws.amazon.com/about-aws/whats-new/2026/01/change-the-server-side-encryption-type-of-s3-objects/

Official AWS Documentation for Updating server-side encryption for existing data.

---

# I Built TrumpDaily to Track Donald Trump Without the Noise

URL: https://ercan.cloud/https-trumpdaily-site-i-built-a-news-aggregator-so-i-can-finally-keep-up/
Date: 2026-01-22T23:11:48+03:00

Look, tracking Trump is exhausting. The man dominates headlines across BBC, Guardian, NPR, Al Jazeera, and even Babylon Bee. Checking 10+ ...


Look, tracking Trump is exhausting. The man dominates headlines across BBC, Guardian, NPR, Al Jazeera, and even Babylon Bee. Checking 10+ sites daily? No thanks. So I built Trump Daily! It's a self-hosted RSS aggregator that dumps everything into one clean interface. https://trumpdaily.site was born! 

And honestly? This was the most fun I've had coding in months a few years ago.

The Stack (Or: Why I Chose Boring Tech)

Backend: Python/Flask (yes, I love it!)

Database: PostgreSQL

Cache: Redis (for literally everything)

Tasks: Celery (background RSS fetching)

Frontend: Vanilla JS (no React, fight me)

Deploy: Docker Compose (one command, done)

The whole thing is ~1,000 lines of Python and 400 lines of JS. Runs locally. No tracking (Just Google Analytics). No cloud bills. No bullshit.

How It Works

RSS Feeds → Celery → Keywords → PostgreSQL → Flask API → Browser
                         ↓
              (Funny? Political? Serious?)

Every 5 minutes, Celery fetches feeds and categorizes articles using keyword matching:

"investigation" + "indictment" = Serious

"ridiculous" + "bizarre" = Funny

"tariff" + "trade war" = Economic

Simple regex, ~85% accuracy. No ML needed.

Why This Was Ridiculously Fun

1. It Actually Works
Unlike most side projects that collect dust, I use this every morning. Building something you genuinely need hits different.

2. Vanilla JS is Liberating
No webpack. No babel. No 500MB node_modules. Just 400 lines of JavaScript that load instantly. The frontend filters 500+ articles client-side without breaking a sweat.

3. Docker Compose is Magic
One docker compose up -d spins up 5 services (nginx, flask, postgres, redis, celery). Zero configuration hell.

4. Python Gets Shit Done
Weekend project → Working prototype in 3 days. The ecosystem (feedparser, SQLAlchemy, Celery) just works. Zero library bugs.

The Trade-Offs (Or: What If I Used Go/Rust?)

Python's not fast. Let's be real:

LanguageRSS ParsingMemoryDev TimePython15-30s~600MB3 daysGo3-5s~50MB5-7 daysRust2-4s~30MB9-15 daysNode.js12-25s~500MB3 daysBun5-8s~200MB3 days
Could Go parse feeds 10x faster? Yes.
Would it matter? Hell no.

The app already feels instant. RSS updates every 5 minutes. Who cares if ingestion takes 15 seconds vs 3 seconds? I'm not running Twitter here.

When I'd Choose Differently

10,000 concurrent users? Go or Rust.

Real-time WebSockets? Elixir/Phoenix.

$5/month VPS? Go (lower memory).

Already know Rust? Use what you know.

But for a single-user, self-hosted tool? Python is perfect.

Best Decisions

- Client-side filtering - Load all 500 articles, filter in browser. Fast and simple.
- Docker Compose - No Kubernetes complexity for a personal project.
- No frameworks - Vanilla JS loads in milliseconds, no build step.
- PostgreSQL - Proper indexes, no SQLite locking drama.
- Privacy-first - Runs locally, zero tracking, zero cloud.

Worst Decisions

- No full-text search - PostgreSQL LIKE is basic, but good enough.
- No real-time updates - Articles refresh every 5 min, not live.
- Memory usage - 600MB is overkill for this, but who cares.

Want to Contribute?

The repo is private on GitHub, but I'm happy to add collaborators. If you want to:

Explore the code

Contribute features

Fork it for your own aggregator

Just send me your GitHub username or your email address. I kept it private to avoid spam, but I love collaboration.

Lessons Learned

Boring tech wins - Flask, PostgreSQL, Redis are boring. They're also battle-tested and predictable.

Client-side rendering is underrated - Modern browsers are fast. Use them.

Docker Compose > Kubernetes - For small projects, K8s is insane overkill.

Vanilla JS is enough - React/Vue are great, but unnecessary complexity for simple UIs.

RSS isn't dead - Simple, standardized, no API keys, just works.

The Verdict

Python was absolutely the right choice. Not because it's fast (it's not), but because:

I shipped in 3 hours with Github Co-Pilot

The code is readable

Performance is "good enough"

I actually enjoy maintaining it

Could Go be 10x faster? Sure. Would it matter? Nope.

Pick tools that let you ship fast, solve the problem, and don't over-engineer.

Tech: Python, Flask, PostgreSQL, Redis, Celery, Vanilla JS
Lines of Code: ~1,400 total
Dev Time: 3 hour with Github Co-Pilot
Deploy: docker compose up -d
Status: Private repo, collaborators welcome

2026. Built it, shipped it, use it daily. This is how side projects should feel.

---

# When Spotify’s Share-to-Instagram Flow Turns Into a Free Billboard

URL: https://ercan.cloud/when-spotifys-share-to-instagram-flow-turns-into-a-free-billboard/
Date: 2025-12-09T01:11:36+03:00

I dug into how Spotify generates story assets and wrote up the possible attack vectors + where the metadata pipelines might be vulnerabi...


Earlier this week, I tried sharing a song on Instagram Stories  "Füsun Önal – Ah Nerede", the 2004 release. 

Spotify → Share → Instagram. Something we all do a thousand times.

But instead of the album cover, Instagram opened with a completely unrelated person’s Instagram profile screenshot. Not mine. Not Spotify’s. Someone else’s essentially a free ad.

And it didn’t look like a glitch. It looked injected.

No suspicious apps. No device compromise. The only thing out of the ordinary was that the “share image” Spotify generates had been swapped.

This might sound like a tiny oddity. It isn’t. It’s a signal that a trusted cross-platform media pipeline might be vulnerable.

How Could This Happen?

Below are the most realistic paths, based on how Spotify structures share assets and how Instagram ingests third-party media.

1. Share Asset Injection via a Manipulated Media Endpoint

Spotify doesn’t screenshot your screen; it generates a dynamic image server-side.
If someone gains the ability to override the image URL or the CDN asset associated with that track, they can push their own image into every share attempt.

This could happen through:

compromised distributor metadata

misconfigured catalog ingestion

legacy asset references Spotify still honors

CDN cache poisoning at the object level

For older catalog tracks (especially pre-Spotify era imports), this is surprisingly plausible.

2. Metadata Poisoning on Legacy Tracks

Tracks published long before streaming often carry messy metadata.
If an attacker slipped in a malformed image, link, or social-share-template fields during a re-upload or distribution update, Spotify’s pipeline might accept it without proper sanitization.

This is classic “trusting legacy fields too much.”

3. Abusing an Old Content Pipeline Still in Production

Spotify imported millions of tracks using tools built years ago. Some of those ingestion paths still exist for label/distributor uploads.

If someone discovered that older pipelines don’t sanitize image fields, they could attach arbitrary assets that propagate to the share layer.

That’s free real estate.

4. A Real-World Vulnerability Test

There’s also a non-zero chance that someone tested whether Spotify’s share-to-Instagram pipeline can be hijacked.

And at least in this one case, it worked.

Why This Is a Security Issue (Not Just a Funny Glitch)

A compromised share flow gives attackers:

Instant reach every user who shares the song unknowingly spreads the attacker’s content

Zero friction delivery no warnings, no approvals, no prompts

Trusted placement Instagram Stories feel “safe”

Perfect amplification even a tiny percentage of hijacked shares equals thousands of free impressions

And if someone decides to escalate:

phishing images

QR codes

brand impersonation

fake giveaways

crypto scams

This becomes a social engineering goldmine.

Technical Breakdown: Where the Weak Link Probably Is

Spotify’s Flow

User taps Share.

Spotify generates or fetches a preview asset.

Spotify sends a structured intent + image to Instagram.

Instagram imports it into a Story template.

If step 2 is compromised, everything downstream inherits the malicious asset.

Instagram’s Flow

Instagram trusts the media Spotify sends.
There is no verification mechanism for “Is this image really representing this track?”

That trust is the entire vulnerability.

Most Likely Weak Spot

Old catalog entries stored with unusual or malformed metadata that Spotify’s new pipeline still respects.
Something like:

social_share_image_url = https://attackercdn.com/injected.jpg

If Spotify doesn’t validate that URL at generation time, the door is open.

Mitigation Ideas Spotify Should Probably Consider

1. Strict Source Validation

Only allow artwork from verified, whitelisted storage buckets; not arbitrary distributor metadata.

2. Regenerate Legacy Assets

Force all old catalog items to use Spotify-generated assets rather than distributor-provided links.

3. Metadata Sanitization

Reject any metadata fields that point to external URLs unless cryptographically verified.

4. Share Asset Integrity Tokens

Instagram could require integrity metadata (such as a checksum or signature) to verify the asset. This is how you stop silent injection.

Why This Could Go Viral

Touches Spotify, Instagram, and nostalgia.

Mixes a weird glitch with a real security implication.

People love “Look, the share button can be hacked for free ads” stories.

The exploit path feels clever enough to spark debate.

It exposes how old metadata quietly exists inside modern platforms.

It’s the perfect weekend Hacker News thread bait.

The Bigger Point

This whole story highlights something we rarely think about:

Modern social features are built on top of legacy metadata nobody has audited in years.

And if someone can hijack the share pipeline of a global music platform with a single track’s metadata…

…we don’t have a bug, we have an attack surface.

P.S. I don't usually write about this kind of thing, but it felt really good to think about it and write about it. I'd love to hear your thoughts. If you'd like to reach me, you can use the contact form or my LinkedIn profile at https://linkedin.com/in/ercanermis.

Thank you so much for taking the time to read this. I'm so glad you're here.

---

# AWS Monthly (Nov '25) The Stateful Serverless Revolution

URL: https://ercan.cloud/aws-monthly-nov-25-the-stateful-serverless-revolution/
Date: 2025-11-30T20:21:43+03:00

November and re:Invent 2025 brought us the "holy grail" of serverless: AWS Lambda Durable Functions. This feature introduces...


November and re:Invent 2025 brought us the "holy grail" of serverless: AWS Lambda Durable Functions.

This feature introduces "Stateful Serverless" directly into the Lambda runtime. By using the new withDurableExecution wrapper, you can now write long-running workflows that persist for up to one year. When your function hits a context.wait() call, the compute is suspended and you stop paying, until the event or human approval resumes it.

It effectively replaces many complex Step Functions use cases with pure, testable code. We also got Graviton5, which is 25% faster than Graviton4 and features the Nitro Isolation Engine. This engine uses formal verification to provide mathematical proof of workload isolation.

November proved that the line between "Serverless" and "Infrastructure" is officially gone. Everything is just "Intent" now.

---

# When the Cloud Sneezes, the World Catches a Cold - Lessons from the us-east-1 Meltdown

URL: https://ercan.cloud/when-the-cloud-sneezes-the-world-catches-a-cold-lessons-from-the-us-east-1-meltdown/
Date: 2025-10-20T10:05:41+03:00


Today, once again, half the internet went dark not because of a global power failure or a cyber-attack, but because a single AWS region, us-east-1 (N. Virginia), had a bad day.

Perplexity went down. Amazon itself stumbled. Substack, Signal, Fortnite, and countless others vanished into the ether. Even services like Statuspage.io, used by companies to tell their customers “we’re down!”, couldn’t update their own status pages, because they, too, rely on the same broken backbone. It’s the definition of irony.

And it’s also the definition of a single point of failure world.

The Illusion of “Global” Cloud

We like to think the internet is distributed, that it’s everywhere, resilient, unstoppable. But when a data center in Virginia sneezes, Europe, Asia, and Africa catch pneumonia.
That’s because, in reality, the “global” web runs through a handful of U.S. cloud regions, most notably AWS us-east-1, which quietly powers much of the internet’s backbone services.

It’s absurd when you think about it: billions of people’s digital lives hang on one physical location in one country, maintained by one company.

Globalization has, ironically, created the least resilient system imaginable, everything centralized, interconnected, and entirely dependent on a few mega-providers.

Monopoly Clouds and the Fragility of Scale

AWS, Google Cloud, and Azure have grown so large that they’ve become too big to fail yet too fragile to trust.
When they work, they’re beautiful. When they fail, the world stops.

And for all the billions they charge, we’re still here watching mission-critical systems crumble because a DNS issue in Virginia decided it was done for the night. These providers charge a premium, but when it comes to reliability and accountability, the value proposition starts to feel like a bad joke.

Let’s be honest, they don’t deserve the price tags they carry if the result is global chaos caused by a regional hiccup.

The Case for a Hybrid Future

Here’s the uncomfortable truth: companies that moved everything to the cloud have made themselves hostages to these monopolies.
A more innovative, more sustainable approach is to flip the model:

Run your own physical datacenter (central region) for production workloads.

Use the cloud for secondary, backup, and disaster recovery.

That way, when the next “cloudquake” hits, your business doesn’t collapse with it.

Imagine a world where every office, every building, every organization runs its own micro-datacenter, small, efficient, locally managed.
Cloud providers could still play a part, offering affordable hardware leasing, long-term equipment rental with buy-out options, and easy cloud-to-local sync solutions.

This hybrid ecosystem would be resilient, distributed, and more human.

The Human Side of Infrastructure

Here’s another point we shouldn’t forget: AI won’t replace the engineers who build and maintain these systems.
Datacenter technicians, network admins, sysadmins, cloud ops, and DevOps engineers are the people who keep the digital world breathing. The more decentralized our systems become, the more skilled humans we’ll need to keep them alive.

So instead of chasing an AI-only fantasy, let’s plan for real infrastructure with real people behind it.

A Wake-Up Call

Today’s AWS outage isn’t just an operational blip; it’s a wake-up call.

A reminder that the cloud is not the sky, it’s just someone else’s computer, sitting in a building in Virginia, running on hardware that breaks, DNS that misbehaves, and humans who make mistakes.

If we want a truly global and resilient internet, we need to stop worshipping monopolies and start rebuilding independence.
Because when the cloud sneezes, we shouldn’t all have to hold our breath.

Written with a bit of sarcasm, a lot of frustration, and a deep love for resilient architecture.
- Ercan Ermis

---

# AWS Monthly (Sep '25): Vega OS &amp; eBPF Observability

URL: https://ercan.cloud/aws-monthly-sep-25-vega-os-ebpf-observability/
Date: 2025-09-30T20:16:41+03:00

In a surprise move, AWS released Vega OS in September. Vega is a specialized, Linux-based OS optimized for the edge and high-performance UI...


In a surprise move, AWS released Vega OS in September. Vega is a specialized, Linux-based OS optimized for the edge and high-performance UI rendering (with a React Native core). It’s ultra-lightweight and designed to boot in milliseconds, perfect for the next generation of smart devices.

On the observability side, we got CloudWatch Application Map 2.0, which uses eBPF (Extended Berkeley Packet Filter) for auto-discovery. This allows AWS to map your entire service topology, including un-instrumented legacy services and third-party API calls, without you writing a single line of instrumentation code.

If you have a "spaghetti" architecture where the documentation hasn't kept pace with reality, this tool provides a mathematically accurate map of every packet flow in your environment. September was the month we finally got full transparency into our "Shadow IT."

---

# AWS Monthly (Aug '25): Big Data, Zero Effort

URL: https://ercan.cloud/aws-monthly-aug-25-big-data-zero-effort/
Date: 2025-08-31T20:14:51+03:00

August was all about "Data Gravity." Amazon Aurora storage limits jumped to 256 TiB, which effectively ends the "sharding" conversation for...


August was all about "Data Gravity." Amazon Aurora storage limits jumped to 256 TiB, which effectively ends the "sharding" conversation for 99.9% of companies.

However, the real star was the expansion of Zero-ETL Integrations. AWS enabled seamless, near-real-time replication from Aurora to OpenSearch and RDS to Redshift. By leveraging the database's internal transaction logs, AWS replicates data to your analytics engine without impacting the source database’s compute.

For data engineers, this means the death of brittle Python/Glue ETL pipelines for simple sync tasks. We also saw DynamoDB get "Attribute-Based Access Control" (ABAC), allowing us to manage permissions via tags rather than complex IAM policies.

August proved that the future of data isn't in moving it, it's in making it accessible exactly where it lives.

---

# AWS Monthly (July '25): Kubernetes at the Edge of Sanity

URL: https://ercan.cloud/aws-monthly-july-25-kubernetes-at-the-edge-of-sanity/
Date: 2025-07-31T20:13:00+03:00

July was a landmark month for the EKS (Elastic Kubernetes Service) crowd. AWS announced that EKS now supports 100,000-node clusters.


July was a landmark month for the EKS (Elastic Kubernetes Service) crowd. AWS announced that EKS now supports 100,000-node clusters.

While 100k nodes might be overkill for most of us, the technical optimizations required to make the Kubernetes control plane stable at that scale benefit everyone. The scheduler is faster, and ETCD performance is more resilient.

But the real game-changer was EKS Auto Mode. This is the "No-Ops" evolution of Kubernetes. It removes the need to manage Node Groups or even Karpenter. AWS manages the worker plane entirely, selecting the best instance types based on pod requirements in real time.

If you've been avoiding Kubernetes because of the "operational tax," July was the month that tax was significantly reduced. It’s the closest we’ve ever been to a truly serverless K8s experience.

---

# AWS Monthly (May '25): The Death of the War Room

URL: https://ercan.cloud/aws-monthly-may-25-the-death-of-the-war-room/
Date: 2025-05-31T20:00:00+03:00

May brought CloudWatch Investigations, which have fundamentally changed someone's on-call rotation. Instead of manual log correlation, ...


May brought CloudWatch Investigations, which have fundamentally changed someone's on-call rotation. Instead of manual log correlation, this service uses AI to perform Automated Root Cause Analysis (RCA).

When an alarm triggers, Investigations automatically traces the error. It correlates metric spikes with concurrent events, like a specific Git commit, a Terraform apply, or an RDS parameter change. Instead of a dashboard showing "500 Errors," you get a report saying: "The latency spike in Service A was caused by a configuration change in Service B that triggered a connection leak in RDS."

Combined with CloudWatch RUM Session Replay, you can now visually reproduce the user's journey leading up to a crash. It’s effectively a "Time Machine" for your infrastructure. If you're still doing manual log-diving during incidents, May gave you back your weekends.

---

# Automating AWS CloudWatch Log Group Tagging with Python and Boto3

URL: https://ercan.cloud/automating-aws-cloudwatch-log-group-tagging-with-python-and-boto3/
Date: 2025-04-16T08:01:03+03:00


Managing tags for AWS CloudWatch log groups is crucial for operational visibility, cost management, and effective resource organization. Tagging log groups manually can be cumbersome, especially when dealing with a large number of log groups. This article outlines a straightforward method to automate this task using Python and the AWS SDK for Python (Boto3).

Importance of Automating CloudWatch Log Group Tagging

Automation ensures:

Consistent tagging across your AWS resources.

Reduced manual effort and human errors.

Enhanced ability to track costs and usage accurately.

Prerequisites

Python 3 installed

AWS CLI configured with appropriate permissions

Boto3 (pip install boto3)

IAM permissions:logs:DescribeLogGroups

logs:ListTagsLogGroup

logs:TagLogGroup

Python Script for Tagging CloudWatch Log Groups

Below is a Python script that automatically applies specific tags to AWS CloudWatch log groups that currently have no tags.

Python Script

import boto3

AWS_REGION = 'us-east-1'  # Replace with your region

# Tags to apply
TAGS_TO_APPLY = {
    'deployed_via': 'manual',
    'env_name': 'development',
    'projectid': 'ID-12345',
}

def tag_cloudwatch_log_groups():
    logs_client = boto3.client('logs', region_name=AWS_REGION)

    paginator = logs_client.get_paginator('describe_log_groups')
    for page in paginator.paginate():
        for log_group in page&#91;'logGroups']:
            log_group_name = log_group&#91;'logGroupName']
            existing_tags = logs_client.list_tags_log_group(logGroupName=log_group_name).get('tags', {})

            if not existing_tags:
                print(f"Adding tags to log group '{log_group_name}'")
                logs_client.tag_log_group(
                    logGroupName=log_group_name,
                    tags=TAGS_TO_APPLY
                )
            else:
                print(f"Log group '{log_group_name}' already has tags. Skipping.")

if __name__ == '__main__':
    tag_cloudwatch_log_groups()

This script will iterate through all your CloudWatch log groups, applying tags only to log groups that currently have none.

Advantages

Simplified resource management

Enhanced cost allocation and auditing

Streamlined operational tasks

Conclusion

Automating CloudWatch log group tagging using Python and Boto3 simplifies administrative tasks and helps maintain a well-organized and cost-effective AWS environment.

---

# Automating AWS ECR Tagging with Python and Boto3

URL: https://ercan.cloud/automating-aws-ecr-tagging-with-python-and-boto3/
Date: 2025-04-15T09:03:02+03:00


Proper tagging of AWS resources is critical for efficient resource management, cost allocation, and auditing. If you have numerous AWS Elastic Container Registry (ECR) repositories, manual tagging can be tedious and error-prone. This article provides a simple and effective solution: automating the tagging of ECR repositories using Python and the AWS SDK for Python (Boto3).

Why Automate Tagging?

Automating tagging saves time, ensures consistency, and prevents costly mistakes. Tags help track resources related to specific projects, environments, or cost centers.

Prerequisites

Python 3 installed

AWS CLI installed and configured

Boto3 library installed (pip install boto3)

IAM permissions:ecr:DescribeRepositories

ecr:ListTagsForResource

ecr:TagResource

Tagging AWS ECR Repositories with Python

Here's a straightforward Python script using Boto3 to add predefined tags to all your ECR repositories that currently have no tags.

Python Script

import boto3

AWS_REGION = 'eu-central-1'  # Replace with your region

# Define your tags
TAGS_TO_APPLY = &#91;
    {'Key': 'provisioned', 'Value': 'manual'},
    {'Key': 'environment', 'Value': 'dev'},
    {'Key': 'project', 'Value': 'Project_Name'},
]

def tag_ecr_repositories():
    ecr_client = boto3.client('ecr', region_name=AWS_REGION)

    paginator = ecr_client.get_paginator('describe_repositories')
    for page in paginator.paginate():
        for repo in page&#91;'repositories']:
            repo_arn = repo&#91;'repositoryArn']
            repo_name = repo&#91;'repositoryName']

            existing_tags = ecr_client.list_tags_for_resource(resourceArn=repo_arn).get('tags', &#91;])

            if not existing_tags:
                print(f"Adding tags to repository '{repo_name}'")
                ecr_client.tag_resource(resourceArn=repo_arn, tags=TAGS_TO_APPLY)
            else:
                print(f"Repository '{repo_name}' already has tags. Skipping.")

if __name__ == '__main__':
    tag_ecr_repositories()

The script will automatically scan through your ECR repositories and add tags only to those repositories that currently have no tags.

Benefits

Saves valuable administrative time.

Ensures consistency across your AWS environment.

Reduces the risk of human error.

Conclusion

Automating the tagging of AWS resources is an essential practice for effective cloud management. Using Python with Boto3 simplifies this task significantly, ensuring your AWS resources remain organized and easily manageable.

---

# Automating ECR Image Cleanup with Bash

URL: https://ercan.cloud/automating-ecr-image-cleanup-with-bash/
Date: 2025-04-11T12:40:42+03:00


Managing container images in Amazon ECR (Elastic Container Registry) is crucial for keeping your registry clean and cost-effective. Over time, unused or deprecated images can accumulate, potentially leading to increased storage costs and operational overhead. One common scenario is removing images that follow a specific tagging pattern, in this case, any image tagged with versions following the “9.x.x” format, where x represents one or more digits.

This article introduces a Bash script designed to automate the cleanup of ECR images. The script provides two main modes, dry-run and apply, allowing you to simulate or execute deletions based on your needs.

Script Overview

The Bash script leverages the AWS CLI and jq (a lightweight and flexible command-line JSON processor) to:

List All ECR Repositories:
It queries AWS ECR for all available repositories in your account.

Fetch Images and Tags:
For each repository, it lists image details (including image digests and tags).

Apply Tag Filtering:
It uses a regular expression (^9\.[0-9]+\.[0-9]+$) to find images whose tags match the "9.x.x" format (e.g., "9.0.1" or "9.12.3").

Operational Modes:Dry Run (--dry-run): The script only lists the images that would be deleted, offering an opportunity to verify which images qualify for deletion without any risk.

Apply (--apply): It goes ahead and deletes the identified images from the repositories.

Usage Information:
If the script is executed without a valid argument, it outputs a helpful usage message explaining the available options.

Key Features

1. Automatic Repository Discovery

What It Does:
The script dynamically retrieves all available ECR repositories rather than requiring manual input or hardcoding repository names.

Benefit:
This ensures that the script remains scalable and adaptable, handling changes in your account’s repository list without extra configuration.

2. Flexible Tag Filtering with Regex

What It Does:
Uses a regular expression to match version tags that start with 9. followed by two more groups of digits separated by a period.

Benefit:
The regex-based approach provides precise control over which images should be targeted. It allows for maintaining a naming convention and cleaning up images that fit a specific outdated or deprecated versioning pattern.

3. Dual-Mode Operation (Dry-Run vs. Apply)

What It Does:
The script accepts command-line arguments to either simulate the deletion (--dry-run) or actually execute it (--apply).

Benefit:
This dual-mode approach provides safety by allowing you to verify the impact before performing any destructive actions. It’s a valuable feature for administrators who need to test scripts in production-like environments first.

4. Robust Error Handling and Strict Mode

What It Does:
The script uses Bash’s set -euo pipefail to enforce strict error checking. This setting causes the script to stop execution when errors occur, when referencing unset variables, or when any command in a pipeline fails.

Benefit:
This minimizes the risk of incomplete operations or unpredictable script behavior, promoting a more reliable execution environment.

5. Clear Usage Documentation

What It Does:
If an incorrect or no argument is supplied, it prints a help message with clear usage instructions and descriptions of each mode.

Benefit:
This helps reduce user error and clarifies how to operate the script, making it more accessible even for less experienced users.

Pros and Cons

Pros

Safety and Predictability:
The inclusion of a dry-run mode helps administrators preview changes before committing to deletion, reducing the risk of accidental data loss.

Automated Discovery:
Automatic querying of all repositories simplifies administration and ensures comprehensive cleanup.

Regex-Based Filtering:
A robust regular expression for tag filtering makes it easy to target specific images based on version patterns.

Simplicity and Extensibility:
Written in Bash, the script is both simple to understand and modify. Its straightforward structure makes it accessible for further customization to suit additional needs.

Robust Error Handling:
The script’s error-handling mechanisms (using set -euo pipefail) enhance reliability by halting execution on unexpected errors.

Cons

Dependency on AWS CLI and jq:
The script requires both the AWS CLI and jq to be installed and configured correctly. In environments where these tools aren’t pre-installed, extra setup is necessary.

Limited Regex Flexibility:
The current regex is specifically tuned to match "9.x.x" versions. If your versioning scheme changes, you may need to adjust the regex accordingly, making the script less flexible unless updated.

Potential for Over-Deletion:
If an image is tagged with multiple values and one of them matches the regex, the deletion command will act on that image. Extra caution is necessary when using the deletion mode in a production environment.

No Rollback Mechanism:
Once an image is deleted (in --apply mode), recovery is not straightforward. Administrators need to have backup or version control measures in place if deletion was accidental.

Verbose Logging:
The script prints messages for each image processed, which might be overwhelming in environments with a large number of images. This could potentially be refined for a more concise logging approach.

Best Practices and Recommendations

Test in a Staging Environment First:
Always run the script in --dry-run mode on a staging environment to verify that it correctly identifies the target images.

Implement Backups:
Ensure that images are backed up or versioned appropriately if there’s a risk of needing to roll back any changes.

Monitor Logs:
If you intend to run this script periodically (e.g., via cron), consider redirecting output logs to a file and implementing log rotation.

Regular Updates:
Maintain your script by reviewing and testing it whenever your ECR usage or image tagging strategy changes.

Fine-Tune Regex When Needed:
Adjust the regex pattern if your versioning system evolves or if you introduce new tagging conventions.

The Script

#!/bin/bash
# This script processes all ECR repositories and their images, checks for image tags matching the pattern "9.x.x"
# (where x represents one or more digits), and then either lists (dry run) or deletes (apply) those images.
#
# Usage:
#   ./cleanup_ecr.sh --dry-run    # Lists images that would be deleted without performing deletion.
#   ./cleanup_ecr.sh --apply      # Deletes images with tags matching "9.x.x".
#   ./cleanup_ecr.sh              # Displays this help message.

set -euo pipefail

function usage() {
    echo "Usage: $0 &#91;--dry-run | --apply]"
    echo ""
    echo "  --dry-run   List images that match the tag pattern '9.x.x' without deleting them."
    echo "  --apply     Delete images that match the tag pattern '9.x.x'."
}

# Check command-line arguments
if &#91; "$#" -ne 1 ]; then
    usage
    exit 1
fi

MODE="$1"
if &#91;&#91; "$MODE" != "--dry-run" && "$MODE" != "--apply" ]]; then
    usage
    exit 1
fi

# Define regex pattern for tags: "9." followed by digit(s) a dot then digit(s)
REGEX="^9\.&#91;0-9]+\.&#91;0-9]+$"

echo "Mode: $MODE"
echo "Fetching list of ECR repositories..."

# Retrieve all repository names.
repositories=$(aws ecr describe-repositories --query 'repositories&#91;].repositoryName' --output text)

# Check if any repositories were returned.
if &#91; -z "$repositories" ]; then
    echo "No ECR repositories found."
    exit 0
fi

# Process each repository.
for repo in $repositories; do
    echo "------------------------------"
    echo "Processing repository: $repo"
    
    # Retrieve image details: digest and tags.
    images_json=$(aws ecr describe-images \
        --repository-name "$repo" \
        --query 'imageDetails&#91;*].{Digest: imageDigest, Tags: imageTags}' \
        --output json)
    
    # Determine the number of images in this repository.
    image_count=$(echo "$images_json" | jq 'length')
    if &#91; "$image_count" -eq 0 ]; then
        echo "  No images found in repository $repo."
        continue
    fi

    # Iterate over each image.
    for (( i=0; i<image_count; i++ )); do
        image_digest=$(echo "$images_json" | jq -r ".&#91;$i].Digest")
        # Extract tags; some images might have no tags.
        tags=$(echo "$images_json" | jq -r ".&#91;$i].Tags&#91;]" 2>/dev/null || true)

        if &#91; -z "$tags" ]; then
            continue
        fi
        
        # Loop through each tag.
        for tag in $tags; do
            if &#91;&#91; $tag =~ $REGEX ]]; then
                echo "Match found - Repo: '$repo', Digest: '$image_digest', Tag: '$tag'"
                if &#91; "$MODE" == "--apply" ]; then
                    echo "  Deleting image..."
                    aws ecr batch-delete-image \
                        --repository-name "$repo" \
                        --image-ids imageDigest="$image_digest",imageTag="$tag"
                fi
            else
                echo "Skipping - Repo: '$repo', Digest: '$image_digest', Tag: '$tag'"
            fi
        done
    done
done

echo "Process completed."

Conclusion

The provided Bash script offers a robust and practical solution for automating the cleanup of Amazon ECR images that match a specified tag pattern. With its clear separation between simulation (dry-run) and execution (apply) modes, along with its automated repository discovery and error-handling mechanisms, the script is a powerful tool for container image management.

However, administrators must also be aware of its dependencies and carefully tailor the regex filter to align with their specific versioning scheme. By following best practices, such as testing in a staging environment and monitoring deletion logs, the script can significantly streamline ECR maintenance, reduce storage bloat, and improve overall operational efficiency.

---

# Update ECR Repositories with Bash Script

URL: https://ercan.cloud/update-ecr-repositories-with-bash-script/
Date: 2025-04-08T10:05:29+03:00


Below is an example Bash script that uses the AWS CLI to retrieve all your Amazon ECR repositories and then sets the image tag mutability of each repository to MUTABLE. Before running the script, ensure you have the AWS CLI installed and configured with appropriate permissions.

#!/bin/bash
# This script fetches all Amazon ECR repositories and sets their image tag mutability to MUTABLE.

# Fetch all repository names from ECR.
repositories=$(aws ecr describe-repositories --query "repositories&#91;].repositoryName" --output text)

# Check if any repositories were found.
if &#91; -z "$repositories" ]; then
  echo "No ECR repositories found."
  exit 1
fi

# Loop over each repository and update its image tag mutability.
for repo in $repositories; do
  echo "Setting image tag mutability to MUTABLE for repository: $repo"
  aws ecr put-image-tag-mutability --repository-name "$repo" --image-tag-mutability MUTABLE | tee
  
  if &#91; $? -eq 0 ]; then
    echo "Successfully updated $repo"
  else
    echo "Failed to update $repo"
  fi
done

Explanation

Fetching repositories:
The script starts by running aws ecr describe-repositories to retrieve a list of repository names using a JMESPath query. The --output text option formats the output as plain text.

Check for no repositories:
It checks if the $repositories variable is empty. If no repositories are found, it exits with a message.

Loop and update mutability:
For each repository name, the script calls aws ecr put-image-tag-mutability setting the --image-tag-mutability flag to MUTABLE.
A success or failure message is printed based on the exit status of each command.

Permissions and AWS CLI configuration:
Make sure your AWS CLI is correctly configured (for example, via aws configure) with credentials that have permissions to perform describe-repositories and put-image-tag-mutability actions in ECR.

This script automates the process of ensuring all repositories are mutable when it comes to image tags, making future updates or tagging changes possible.

---

# Why Automated Tests Are Essential in Your CI/CD Pipeline and Development Flow

URL: https://ercan.cloud/why-automated-tests-are-essential-in-your-ci-cd-pipeline-and-development-flow/
Date: 2025-04-06T06:44:11+03:00


Let’s talk about a scenario we’ve all been in: you’ve just wrapped up a shiny new feature, you’re excited to merge it in, and, boom, something breaks in production that you didn’t catch in your local environment. That’s the kind of horror story we want to avoid. This is exactly where automated tests come into play, especially within your continuous integration and continuous delivery (CI/CD) pipeline.

In the screenshot above, you can see an example GitHub Actions workflow running Playwright tests. We’re spinning up an EC2 instance, running end-to-end (E2E) tests, and then shutting down the instance. This workflow is a classic example of how automation can streamline your development cycle and keep your application stable. Let’s dive into why these tests are so important and how they fit into a healthy development flow.

1. Early Detection of Bugs

One of the biggest advantages of having tests in your pipeline is that they catch issues early, often before they ever hit production. If you wait until you’ve merged your code into the main branch to discover a bug, it could be hours (or days) before you realize something is wrong. And by then, other features might have been layered on top, making it a nightmare to unravel. Automated tests, triggered by every push or pull request, ensure that if something goes off the rails, you’ll know immediately.

a. Saves Time and Resources

Let’s face it, manually testing your entire application is tedious and error-prone. If you rely on manual testing alone, there’s a good chance you’ll miss subtle regressions. Automated tests free you from repetitive checks and let you focus on new features and more interesting challenges. As soon as the pipeline flags a problem, you know exactly where to start looking.

2. Confidence in Deployment

When you have a robust suite of automated tests, you can deploy with a lot more confidence. Instead of crossing your fingers every time you push to production, you can rely on your test suite to tell you if something is fundamentally broken. This is especially critical for large teams, where multiple developers are merging changes into the same codebase.

a. Continuous Integration and Continuous Delivery

CI/CD isn’t just a buzzword. It’s a development philosophy that encourages smaller, more frequent releases. Automated tests are the backbone of CI/CD. Without them, you’re taking a leap of faith every time you release. With them, you get real-time feedback about the stability of your code.

b. Seamless Collaboration

When your CI/CD pipeline runs tests automatically, everyone on the team benefits. Each developer knows their changes are tested in the same environment, under the same conditions. This consistency reduces the “it works on my machine” problem and fosters a culture of shared responsibility for code quality.

3. Regression Prevention

It’s not just about new features. Every code change has the potential to break something that was previously working. We call these issues regressions, and they’re notoriously difficult to spot if you don’t have a solid suite of automated tests. End-to-end tests, like the Playwright tests shown in the screenshot, are particularly good at catching regressions because they mimic real user interactions from start to finish.

a. Realistic Scenarios

E2E tests let you simulate user actions, clicking buttons, filling out forms, navigating between pages, etc. This means that if a previously functional workflow gets disrupted, your E2E tests will likely catch it. And because they run automatically in the pipeline, you’ll know right away if something regresses.

b. Living Documentation

Think of your tests as living documentation for your application’s functionality. They describe how the system is supposed to behave. When something changes, you update the tests to reflect that new behavior. Over time, you end up with a library of scenarios that help keep your application consistent and predictable.

4. Streamlined Development Flow

Automation isn’t just about preventing catastrophes, it also makes life easier for everyone involved. By integrating tests into your pipeline, you’re creating a frictionless process that allows developers to code with confidence.

a. Faster Feedback Loop

A tight feedback loop is key to an efficient development process. When you push code to a branch and the pipeline instantly tells you if you’ve broken something, you can fix it right away. This immediate feedback reduces context switching and keeps your momentum going.

b. Consistency Across Environments

In the screenshot, you can see the pipeline also involves starting and stopping an EC2 instance. Having a consistent environment for your tests is crucial. If your local machine is configured differently from your staging or production environment, you might miss environment-specific issues. By spinning up the same environment each time, you ensure consistency, which leads to more reliable test results.

5. Scalability and Maintainability

As your application grows, manual testing becomes increasingly unsustainable. A well-structured test suite scales alongside your application. You can add new tests for new features, refactor old tests as needed, and keep everything up to date without blowing your budget or timeline.

a. Easier Onboarding

New developers can jump in more easily when they have a robust test suite to guide them. Tests can serve as a roadmap to your application’s functionality. Plus, if they accidentally break something, the pipeline will catch it before it causes any real damage.

b. Future-Proofing Your Application

Technology moves fast. What works today might be outdated next year. Automated tests give you the freedom to upgrade libraries, frameworks, or even entire architectures, because you can quickly validate that everything still works as intended.

Conclusion

Tests aren’t just a checkbox to tick off, they’re the foundation of a reliable, scalable, and maintainable application. By integrating automated tests into your CI/CD pipeline, you can catch bugs early, prevent regressions, collaborate more effectively, and deploy with confidence. The screenshot of the GitHub Actions workflow running Playwright tests is a snapshot of that process in action: starting an EC2 instance, running end-to-end tests, and shutting it down, all without any manual intervention.

In other words, automated testing within a CI/CD pipeline isn’t just a best practice, it’s practically a superpower for modern software teams. It saves time, reduces stress, and ensures that your end users get the best possible experience. If you’re not already leveraging automated tests in your pipeline, now is the time to start. Your future self (and your users) will thank you.

---

# AWS Monthly (April '25): Cooling the AI Heat

URL: https://ercan.cloud/aws-monthly-april-25-cooling-the-ai-heat/
Date: 2025-03-31T19:59:00+03:00

In April, AWS gave us a peek behind the curtain of their data center engineering. To handle the thermal output of the latest AI clusters, ...


In April, AWS gave us a peek behind the curtain of their data center engineering. To handle the thermal output of the latest AI clusters, they unveiled Direct-to-Chip Liquid Cooling.

This is more than just a hardware curiosity. From a performance standpoint, liquid cooling allows for 3x higher compute density per rack. It ensures that custom silicon like Trainium and Inferentia can maintain peak clock speeds for long-running training jobs without the thermal throttling that often plagues air-cooled environments.

We also saw the launch of EC2 UltraClusters 2.0, which supports non-blocking petabit-scale networking via Elastic Fabric Adapter (EFA). If you're distributed-training models with trillions of parameters, the communication bottleneck between nodes has been virtually eliminated.

April reminded us that "the cloud" is ultimately limited by the laws of physics, and AWS is literally re-engineering the plumbing of the internet to keep up with AI.

---

# AWS Monthly (Mar '25): The Proximity Power Play

URL: https://ercan.cloud/aws-monthly-mar-25-the-proximity-power-play/
Date: 2025-03-31T19:51:00+03:00

Technically, this brings "Single-Digit Millisecond Latency" to almost every major tech hub. If you are building AR/VR backends, indust...


March was about the physical foundation of the cloud. AWS announced a $100 billion global investment, and for us, that translated into over 20 new Local Zones going live.

Technically, this brings "Single-Digit Millisecond Latency" to almost every major tech hub. If you are building AR/VR backends, industrial IoT, or sub-millisecond telemetry systems, the "distance to silicon" just got shorter. We are seeing a move away from "Central Regions" toward a more distributed "Edge-First" architecture.

On the networking side, VPC Lattice received a critical update: Cross Account Trust Labels. This allows you to define service-to-service communication policies based on logical metadata rather than brittle IP ranges or complex peering. It’s a massive win for multi-account governance.

If you haven't looked at VPC Lattice lately, this update makes it the "Service Mesh" of choice for enterprise-scale networking without the operational overhead of Istio.

---

# Streamline Your AWS ECR Management with This Powerful Bash Script

URL: https://ercan.cloud/streamline-your-aws-ecr-management-with-this-powerful-bash-script/
Date: 2025-03-27T06:32:10+03:00


Managing container repositories in AWS ECR (Elastic Container Registry) can quickly become a daunting task, especially as your infrastructure grows. In this article, we’ll take an in-depth look at a handy bash script designed to automate the application of lifecycle policies to your ECR repositories. This script not only simplifies repository management but also ensures that only the most recent images are retained, helping you save on storage costs and keep your registry tidy.

What Does This Script Do?

The primary purpose of this bash script is to automate the process of applying a lifecycle policy that keeps only the latest three images in each AWS ECR repository in a specified region. Here’s a quick overview of its functionality:

Error Handling: The script is configured to exit immediately if any command returns a non-zero status, which minimizes the risk of unintended behavior.

Color-Coded Output: It uses ANSI color codes to highlight messages (such as warnings in red), making the output easier to parse.

Dry-Run Mode: For a safe, non-destructive test, the script includes a --dry-run option that simulates actions without actually applying the policy.

AWS CLI Integration: It retrieves repository names using the AWS CLI and then iterates over each repository to check for an existing lifecycle policy before applying a new one.

Breaking Down the Script

1. Setting Up the Environment

The script starts by enforcing strict error handling with set -e and defines ANSI color codes for enhanced terminal output. This ensures that any error stops the script immediately, preventing unintended operations.

#!/bin/bash
set -e
RED='\033&#91;0;31m'
NC='\033&#91;0m' # No Color

2. Parsing Command-Line Arguments

A simple loop processes command-line arguments to detect if the script should run in dry-run mode. This allows administrators to test the script without making any real changes.

DRY_RUN=false
while &#91;&#91; "$#" -gt 0 ]]; do
    case $1 in
        --dry-run) DRY_RUN=true ;;
        *) echo "Unknown parameter passed: $1"; exit 1 ;;
    esac
    shift
done

3. Specifying the AWS Region and Retrieving Repositories

The script is configured to work in a specified region (eu-central-1 by default). It retrieves the list of repository names from AWS ECR using the AWS CLI. If no repositories are found, it exits with an appropriate message.

REGION="eu-central-1"
repos=$(aws ecr describe-repositories --region "$REGION" --query 'repositories&#91;].repositoryName' --output text)
if &#91; -z "$repos" ]; then
    echo "No repositories found in region $REGION."
    exit 1
fi

4. Defining the Lifecycle Policy

A JSON object is defined within the script to specify the lifecycle policy. This policy is set to keep only the last three images, which is ideal for many use cases where maintaining the most recent images is sufficient.

lifecycle_policy='{
  "rules": &#91;
    {
      "rulePriority": 1,
      "description": "Keep last 3 images",
      "selection": {
        "tagStatus": "any",
        "countType": "imageCountMoreThan",
        "countNumber": 3
      },
      "action": {
        "type": "expire"
      }
    }
  ]
}'

5. Applying the Lifecycle Policy to Each Repository

The script then loops through each repository. It checks if a lifecycle policy already exists. If it does, the repository is skipped, and a warning is shown in red. If not, the lifecycle policy is applied, or a dry-run message is printed if that mode is enabled.

for repo in $repos; do
    echo "Processing repository: $repo"
    if aws ecr get-lifecycle-policy --repository-name "$repo" --region "$REGION" > /dev/null 2>&1; then
        echo -e "${RED}Lifecycle policy already exists for repository $repo. Skipping...${NC}"
    else
        if &#91; "$DRY_RUN" = true ]; then
            echo "DRY RUN: Would apply lifecycle policy to repository: $repo"
            echo "DRY RUN: aws ecr put-lifecycle-policy --repository-name \"$repo\" --lifecycle-policy-text '$lifecycle_policy' --region \"$REGION\""
        else
            echo "Applying lifecycle policy to repository: $repo"
            aws ecr put-lifecycle-policy \
                --repository-name "$repo" \
                --lifecycle-policy-text "$lifecycle_policy" \
                --region "$REGION" | tee
        fi
    fi
done
echo "Processing complete."

Benefits of Using This Script

Automation and Efficiency

Manually applying lifecycle policies across multiple repositories can be tedious and error-prone. This script automates the process, ensuring consistent application of policies and freeing up valuable time for your team.

Cost Management

By keeping only the most recent images, the script helps in managing storage costs. Old, unused images are automatically removed, reducing unnecessary storage expenditure.

Safe Testing with Dry-Run Mode

Before making any permanent changes, you can run the script in dry-run mode. This feature provides a clear preview of what actions will be performed, minimizing the risk of mistakes.

Customizable and Extendable

The script can easily be modified to suit your specific needs, such as changing the retention policy or targeting different regions. Its modular design allows for quick adaptations as your AWS environment evolves.

Conclusion

Automating routine tasks is a key strategy for efficient DevOps and system administration. This bash script for applying lifecycle policies to AWS ECR repositories is an excellent example of how simple automation can drive significant operational benefits. Whether you're looking to reduce costs, streamline your workflow, or enforce a consistent policy across your repositories, this script offers a reliable and customizable solution.

Implement this script in your AWS environment and experience the ease of automated repository management today!

---

# AWS Monthly (Jan '25): Shattering the 6MB Ceiling

URL: https://ercan.cloud/aws-monthly-jan-25-shattering-the-6mb-ceiling/
Date: 2025-01-31T08:36:00+03:00


We kicked off 2025 by addressing one of the oldest "bottleneck" complaints in the serverless community. For years, the 6MB payload limit for synchronous Lambda calls forced us into complex workarounds involving S3 pre-signed URLs or asynchronous patterns for anything remotely data-heavy.

In January, AWS officially bumped Lambda response streaming capabilities to 200 MB. Technically, this is a massive shift in how we handle data egress. By implementing the response-stream content type, your functions can now push massive payloads, high-res media, giant JSON blobs, or real-time AI transcriptions directly to the client. This drastically reduces Time to First Byte (TTFB) because the client starts receiving data as it’s generated, rather than waiting for the entire 200MB buffer to close.

On the storage side, S3 Express One Zone, the high-performance, single-AZ sibling of S3, received a performance profile update. We saw roughly a 15% reduction in latency for small-object GETs. If you’re running high-frequency trading apps or real-time gaming leaderboards, this update effectively lowered your "distance to data."

Engineering Tip: If you're still using the old Invoke API for large reports, migrate to streaming. It simplifies your frontend logic and eliminates the need for "interim" S3 buckets.

---

# Setting up DKIM for Google Workspace (Gmail) using Terraform and AWS Route 53

URL: https://ercan.cloud/setting-up-dkim-for-google-workspace-gmail-using-terraform-and-aws-route-53/
Date: 2024-10-02T05:54:17+03:00

DKIM (DomainKeys Identified Mail) is a critical email authentication technique that helps prevent email spoofing. By using DKIM, you ...


DKIM (DomainKeys Identified Mail) is a critical email authentication technique that helps prevent email spoofing. By using DKIM, you digitally sign your email headers with a private key, and the recipient verifies this signature using your public key, which is stored in the DNS records of your domain. Google Workspace (formerly G Suite) leverages DKIM to ensure the emails sent from your domain are verified as coming from you and not a spammer pretending to be you.

In this article, we’ll dive into how to use Terraform to configure DKIM for Google Workspace, using AWS Route 53 for DNS management. This approach is perfect if you're already managing your infrastructure as code and want to automate the process of creating DNS records for DKIM.

Let's break down the Terraform block you've provided and go step-by-step through what each part does.

1. Overview of the DKIM Process

Before diving into the Terraform code, let’s recap how DKIM works:

Signing Outgoing Emails: Google Workspace signs the outgoing email headers with a private key. The domain owner configures the public key as a DNS record.

Verifying Incoming Emails: The recipient's email server fetches the public key from DNS to verify the signature and ensure that the email has not been tampered with during transit.

To set up DKIM, you’ll create a TXT record in your DNS, containing the public key generated by Google Workspace.

2. Terraform Block Breakdown

The provided Terraform block creates a TXT record in AWS Route 53, which is essential for publishing the DKIM public key for Google Workspace. Here’s a breakdown of each part of the block:

a. Local Variable for DKIM Record

locals {
  dkim_record = "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4FbZGvaSuBW9yDhuswEA6S7QRRZlvvS3Rc6xRK92QJ44+IUoobJf+S3VZGSjdv5Jmpu57JGaq1KsObB5DwObvCW6yhJb6rGd0bgXqTzkfHH7sxnkEbDbNEIyUhxuEbUdWE1KtFz0QMb+GMhxC+j+kntpXtVDWugp9lSa7cnkFVho/gRwVkz47QFD1kQjvP9/QraYZo63M5DObFRFF4rDHvBzNTGl6+FC0OkyA49f19iw1+eFtLWu8BpVIn4HxxZWn04i1KHnm1AX9BztBAoVbJ0ZlF1tEJ5eLMPQg5l0BUcSEYi1RcBKTJU+AJ17fGAJEGr7Ft47SJgHdN9XUPH9NwIDAQAB"
}

Here, the locals block defines a local variable dkim_record. This variable stores the DKIM public key in the format required for DNS.

v=DKIM1: Specifies the version of DKIM being used.

k=rsa: The type of cryptography used; Google typically uses RSA.

p=MIIBIjAN...: The public key itself. This key is generated by Google Workspace and is unique to your domain.

b. Creating the DKIM Record in Route 53

resource "aws_route53_record" "google_domainkey_TXT" {
  zone_id = aws_route53_zone.domain_com.zone_id
  name    = "google._domainkey"
  type    = "TXT"
  ttl     = 300
  records = &#91;
    replace(local.dkim_record, "/(.{255})/", "$1\"\"")
  ]
}

This aws_route53_record resource block is responsible for creating the DNS record in your Route 53 hosted zone. Let’s break down each attribute:

zone_id: This refers to the hosted zone in AWS Route 53 where the record will be added. The aws_route53_zone.domain_com.zone_id references the Route 53 zone for your domain, which Terraform can manage dynamically.

name: The name attribute specifies the DNS name for the record. In this case, it’s set to google._domainkey. This is the specific subdomain that Google Workspace expects for storing DKIM records. The google._domainkey prefix is required by Google to find the public key for verifying DKIM signatures.

type: The type of record is set to TXT. DKIM records are stored as TXT records in DNS.

ttl: The TTL (Time to Live) is set to 300 seconds (5 minutes). This determines how long DNS resolvers should cache the record before fetching it again.

records: This field holds the actual content of the TXT record, which is the DKIM public key. Since DKIM public keys can often exceed 255 characters (the maximum length for a single DNS TXT record), Terraform is using a replace() function to split the string into 255-character chunks.

c. Splitting Long DKIM Records

The replace() function used in this configuration is critical:

replace(local.dkim_record, "/(.{255})/", "$1\"\"")

This line ensures that the DKIM record is split into smaller sections of 255 characters, as DNS servers have a limit on how long a single TXT record can be. The replace() function uses a regular expression to break the string into these chunks and adds the necessary quotes between them.

3. Why Use Terraform for DKIM?

There are a few good reasons to use Terraform for setting up your DKIM DNS records:

Automated DNS Management: With Terraform, you can manage your Route 53 DNS records as code. This makes it easy to version control your DNS configurations, ensure consistency, and reduce manual errors.

Repeatability: Once the Terraform script is written, you can easily apply it across multiple environments or domains. This is especially useful for managing multiple domains or when you need to re-provision infrastructure.

Infrastructure as Code (IaC): Managing infrastructure using code allows for better documentation, collaboration, and auditing. Any changes to your DNS settings are recorded in your version control system (e.g., Git), giving you a clear history of who made changes and why.

4. Steps to Configure DKIM for Google Workspace

To complete the DKIM setup for Google Workspace, you’ll need to follow these steps:

Generate the DKIM key in Google Workspace:Go to your Google Admin Console.

Navigate to Apps > Google Workspace > Gmail > Authenticate Email.

Generate a new DKIM key for your domain and copy the provided public key.

Apply the Terraform Configuration:In your Terraform configuration file, set the dkim_record local variable to the public key generated by Google Workspace.

Run the following commands to apply the configuration to AWS Route 53:

terraform init
terraform plan
terraform apply

This will create the DKIM TXT record in your Route 53 DNS zone.

Enable DKIM Signing in Google Workspace:After the DNS record has propagated (this may take a few minutes), return to the Google Admin Console.

Go back to the Authenticate Email page and click on Start Authentication. This enables DKIM signing for your domain.

5. Verifying Your DKIM Setup

Once DKIM is enabled, you can test whether it’s working by sending an email from your Google Workspace account to a service like DKIMCore’s DKIM validator or other email testing tools. These services will tell you if the DKIM signature is being applied correctly and if the DNS record is set up properly.

Conclusion

Using Terraform to manage DKIM records for Google Workspace and AWS Route 53 streamlines the process of configuring and maintaining your email authentication setup. By automating this with Infrastructure as Code (IaC), you gain the benefits of reproducibility, auditability, and ease of management, all while ensuring your emails are protected from spoofing and other email-based attacks.

With this configuration, you now have a solid DKIM setup that enhances the security of your email communications, ensuring that recipients can trust the messages you send from your domain. Happy Terraforming!

---

# Automate AWS Site-to-Site VPN Monitoring

URL: https://ercan.cloud/automate-aws-site-to-site-vpn-monitoring/
Date: 2024-10-01T06:49:31+03:00

That’s exactly why I created the AWS Site-to-Site VPN Monitoring Script, a tool designed to automate the monitoring of VPN connections ...


In today’s fast-paced, interconnected world, secure and reliable communication between on-premises environments and cloud infrastructures is crucial. For many businesses, AWS Site-to-Site VPN serves as the bridge that connects their data centers to AWS, enabling private, encrypted communication channels. But what happens when your VPN tunnel goes down? Without proper visibility or a notification system in place, downtime can easily go unnoticed, ultimately leading to service disruptions and unhappy customers.

That’s precisely why I created the AWS Site-to-Site VPN Monitoring Script, a tool designed to automate the monitoring of VPN connections and instantly notify teams if any issues arise. In this guide, I’ll walk you through how this script works and why it’s a game-changer for managing your AWS VPN connections.

Why I Created This AWS VPN Monitoring Script 

Managing a cloud infrastructure can be challenging, and when VPN tunnels go down without warning, it can disrupt critical operations. I experienced this firsthand, and as a result, I realized there was a need for a simple, automated solution to monitor VPN tunnels in real time. While AWS offers powerful tools for network security, it doesn’t automatically notify you when a VPN tunnel is down unless you’ve set up an elaborate monitoring system.

This gap led to the creation of the AWS Site-to-Site VPN Monitoring Script. By integrating AWS Lambda, DynamoDB, and Slack/Discord notifications, the script ensures you’re always informed about the status of your VPN tunnels. As a result, there’s no more need for manual checks or worry about unnoticed downtime!

Key Benefits of AWS VPN Monitoring Automation 

1. Real-time Notifications 

When a VPN tunnel goes down, every second counts. This script provides real-time notifications through Slack or Discord, which allows you to take immediate action. Consequently, this quick response minimizes the risk of prolonged outages and keeps your business running smoothly.

2. Historical Status Logging 

The script doesn’t just notify you, it also logs the status of your VPN tunnels into DynamoDB. This enables you to review historical data and gain insights into your network’s reliability over time. For instance, if you want to see when the tunnel went down and for how long, the logs have got you covered.

3. Fully Automated and Reliable 

By running as an AWS Lambda function, this monitoring solution eliminates the need for manual tracking. In addition, the script checks the VPN tunnel statuses regularly and reports any issues automatically. You’ll save both time and effort with this hands-off approach, making the entire process more efficient.

4. Easy Debugging 

Need to troubleshoot? The script includes a debug mode that can be enabled with an environment variable, making it easier to pinpoint any issues or gather more granular information during setup. Moreover, it simplifies the process when something unexpected happens.

Why VPN Tunnel Monitoring is Critical

For many organizations, AWS Site-to-Site VPN connections are essential for secure communication between their on-premises data centers and AWS services. These VPN tunnels are often the backbone of critical operations, such as data replication and hybrid cloud applications. Thus, when a VPN tunnel goes down, the consequences can be severe.

Here’s why you shouldn’t ignore VPN tunnel monitoring:

1. Downtime Costs Money 

Every minute a VPN tunnel is down can result in lost revenue, delayed workflows, and frustrated customers. Proactively monitoring your VPN tunnels helps prevent prolonged outages by alerting you as soon as an issue arises, which means you can act quickly to minimize losses.

2. Security and Compliance Risks 

A broken VPN tunnel that goes unmonitored can expose sensitive data to risks. For industries like finance and healthcare, compliance regulations demand reliable, secure communications. Therefore, monitoring ensures you're alerted immediately if a secure tunnel is compromised, helping you maintain compliance and protect your data.

3. Seamless Operations 

For organizations running hybrid cloud environments, site-to-site VPN connections are critical for maintaining smooth operations. Whether it's syncing data or running distributed applications, an operational VPN tunnel is key to keeping everything running without interruptions. As a result, monitoring these connections ensures uninterrupted operations.

How the AWS Site-to-Site VPN Monitoring Script Works

The script leverages the AWS EC2 DescribeVpnConnections API to retrieve real-time status information about your VPN tunnels. If a tunnel goes down, an alert is sent to your Slack or Discord channel. Additionally, the script logs the VPN tunnel statuses into DynamoDB, providing a record that can be referenced for troubleshooting or auditing purposes later on.

Here’s a breakdown of what the script does:

Monitor VPN tunnels: It checks the status of two tunnels (Tunnel 1 and Tunnel 2) at regular intervals.

Send notifications: It sends alerts via Slack or Discord if either tunnel goes down.

Log tunnel statuses: It stores the status information in a DynamoDB table for historical reference.

How to Get Started with AWS VPN Monitoring

Want to start automating your VPN monitoring? You can easily find the complete project and detailed setup instructions in my GitHub repository: 👉 AWS Site-to-Site VPN Monitoring GitHub Repo

With this script, you’ll minimize downtime, ensure your network’s security and compliance, and, most importantly, avoid missing any critical VPN tunnel alerts.

Conclusion

Automating the monitoring of your AWS Site-to-Site VPN is essential for maintaining the secure and continuous operation of your cloud infrastructure. This script provides peace of mind by offering real-time notifications, historical logging, and easy troubleshooting, all without the need for manual intervention. Furthermore, the automation saves valuable time, allowing you to focus on other critical tasks.

If you’re ready to enhance your cloud network monitoring and reduce the risk of unexpected VPN downtimes, check out the GitHub repository and give it a try! You’ll be better equipped to manage your AWS VPN connections and ensure your business remains online and secure.

By automating your VPN monitoring, you’ll stay ahead of potential issues and keep your operations running smoothly.

---

# Optimizing Docker Images: Tips for Reducing Image Size and Build Time

URL: https://ercan.cloud/optimizing-docker-images-tips-for-reducing-image-size-and-build-time/
Date: 2024-09-27T08:39:25+03:00

Hey, fellow Docker enthusiast! If you're here, you probably love Docker as much as I do, spinning up containers, packaging applications ...


Hey, fellow Docker enthusiast! If you're here, you probably love Docker as much as I do, spinning up containers, packaging applications, and getting everything to run smoothly, no matter the environment. But let's be real for a second: sometimes, Docker images can balloon in size, making builds slow and containers heavier than they need to be. Not exactly what you want, right?

Well, you’re in luck! Today, we’re going to dive deep into optimizing Docker images. By the end of this, you'll know how to shrink those bulky images and speed up your build times without sacrificing performance. Let’s get into it!

Why Docker Image Size Matters

Before we get into the nitty-gritty of optimization, let’s quickly talk about why this matters.

Faster builds: Smaller images mean faster builds, which translates to quicker iteration cycles for you and your team.

Reduced attack surface: A leaner image with fewer layers and dependencies lowers the risk of vulnerabilities creeping into your containers.

Lower deployment costs: Slimmer images use less bandwidth when being pulled to hosts, saving time and money, especially when you're dealing with many instances or CI/CD pipelines.

Efficient use of storage: Your registry (whether Docker Hub, AWS ECR, or something else) will thank you for not hogging space.

Let's Optimize! Top Tips for Reducing Docker Image Size and Build Time

1. Start With a Minimal Base Image

Your base image sets the tone for the rest of your Dockerfile. The official node, python, or ubuntu images are great, but they often come loaded with more than you need.

Pro Tip: Opt for minimal images like alpine. For example, instead of node, use node:alpine. It’s a lightweight alternative that cuts out unnecessary packages.

# Before
FROM node:16

# After
FROM node:16-alpine

With Alpine, you can shrink the image by hundreds of megabytes! Just keep in mind that it might not include some libraries you’re used to, so you may need to manually install those.

2. Use Multi-Stage Builds

Multi-stage builds are a lifesaver for optimizing Docker images. They allow you to separate the build environment (which tends to be large) from the final runtime environment (which should be lean and mean).

Here’s a quick example:

# Step 1: Build Stage
FROM golang:1.19-alpine AS build
WORKDIR /app
COPY . .
RUN go build -o myapp

# Step 2: Final Stage
FROM alpine:latest
WORKDIR /app
COPY --from=build /app/myapp .
CMD &#91;"./myapp"]

In this setup, all the heavy lifting (compiling code, installing dependencies) happens in the first stage, and only the final executable gets copied into the second stage, which is much smaller.

3. Minimize the Number of Layers

Each command in your Dockerfile creates a new layer in the final image, and every layer adds to the overall size. So, combine commands whenever possible to reduce the number of layers.

For example, instead of doing this:

RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git

Combine them into a single layer:

RUN apt-get update && apt-get install -y curl git

By doing this, you reduce the number of layers from three to one.

4. Clean Up After Yourself

Every time you install packages or download files in your image, make sure you clean up afterward. This helps to avoid leaving behind unnecessary files that bloat your image.

For example, if you’re installing packages in an Ubuntu-based image:

RUN apt-get update && \
    apt-get install -y curl git && \
    rm -rf /var/lib/apt/lists/*

Here, the rm -rf /var/lib/apt/lists/* command ensures that the temporary files created during the package installation are removed after the installation is done.

5. Use .dockerignore Effectively

Docker’s equivalent of .gitignore is the .dockerignore file. This file allows you to exclude unnecessary files from your image context (like local development files, node_modules, test folders, etc.), which speeds up build time and reduces image size.

A typical .dockerignore might look like this:

node_modules
.git
.env
logs/
tmp/

Only copy what you truly need into the Docker image to avoid bloating it with local files that aren't relevant in production.

6. Leverage Caching

Docker caches layers during the build process, so take advantage of this to optimize build times. By putting less frequently changing commands (like installing dependencies) at the top of your Dockerfile, you can avoid rebuilding those layers every time.

Example for a Node.js project:

# Install dependencies first (this rarely changes)
COPY package.json package-lock.json ./
RUN npm install

# Copy app files after (this changes more often)
COPY . .

This way, if you change your app files, Docker will only rebuild from the COPY . . step onward, and it will reuse the cached layers for installing dependencies, significantly speeding up builds.

7. Choose the Right Base Image for Your Use Case

Sometimes, it's worth taking a step back to evaluate if you’re using the right base image. For example:

Python applications: You could use python:3.9-slim instead of python:3.9, as the "slim" tag removes unnecessary build tools and libraries.

Go applications: A simple scratch image might be all you need if you're building a static binary. It doesn't get smaller than scratch, which is an empty image.

FROM scratch
COPY myapp /myapp
CMD ["/myapp"]

This approach is perfect for apps where you don’t need a full OS, just the binary.

8. Use Squashing (With Caution)

Docker has a --squash flag that combines multiple layers into one, reducing image size. While not always necessary, it can be useful in certain cases.

You can enable squashing when you build your image like this:

docker build --squash -t yourimage:latest .

However, keep in mind that squashing can make debugging harder because it compresses everything into a single layer. Use it with caution!

Wrapping It Up

Optimizing Docker images doesn’t have to be a complex or tedious task. With a few simple tweaks, like using minimal base images, multi-stage builds, and cleaning up after package installations, you can drastically reduce both image size and build times.

Remember, keeping your images lean isn't just about being efficient; it’s also about speeding up your development workflow, reducing costs, and improving security. So, go ahead and give these tips a try on your next Docker project, you'll notice the difference!

Happy containerizing! 🐳

---

# Monitoring EC2 Disk Space with a Simple Bash Script and Slack Alerts

URL: https://ercan.cloud/monitoring-ec2-disk-space-with-a-simple-bash-script-and-slack-alerts/
Date: 2024-09-22T15:25:41+03:00

In the cloud infrastructure landscape, monitoring the health and resources of EC2 instances is essential. One common challenge for system...


Introduction

In the cloud infrastructure landscape, monitoring the health and resources of EC2 instances is essential. One common challenge for system administrators is ensuring that disk space doesn’t run out, which can lead to performance degradation or service outages. While AWS provides robust monitoring tools like CloudWatch, they can incur additional costs and complexity, especially for smaller or less complex setups.

In this article, we’ll explore a simple and cost-effective solution using a Bash script that runs as a cron job on your EC2 instances. This script will check the disk usage on the instance, and if it exceeds a defined threshold, it will send an alert to a Slack channel. This method requires minimal setup, no external monitoring tools, and keeps costs low.

Why Do You Need Disk Monitoring?

Disk space issues are often overlooked until they lead to problems such as:

Application Failures: Applications or services may fail when there’s insufficient disk space for logging, writing data, or temporary files.

Performance Degradation: Low disk space can cause system slowness, especially for services that rely heavily on read/write operations.

Downtime: Critical systems may crash if essential processes can't operate due to full disks.

Monitoring Disk Space Proactively

Monitoring disk usage helps you avoid such issues by:

Receiving early warnings when disk usage crosses a certain threshold.

Taking preventative actions (like extending the disk size or cleaning up old data) before it becomes critical.

While AWS CloudWatch offers a way to monitor custom metrics like disk space, it incurs additional costs and requires setup and integration with your EC2 instances. A Bash script with a Slack webhook offers a lightweight and low-cost alternative.

The Bash Script Solution

Here is the solution using a Bash script and a Slack webhook. The script checks the disk usage of the root volume (/) and sends an alert to a Slack channel if the usage exceeds a defined threshold. It also includes information like the EC2 instance's hostname, private IP, and public IP to help quickly identify the affected instance.

#!/bin/bash

THRESHOLD=85  # Define the disk usage threshold (in %)
SLACK_WEBHOOK_URL="https://hooks.slack.com/services/your/webhook/url"

# Get EC2 metadata for hostname, public IP, and private IP
hostname=$(curl -s http://169.254.169.254/latest/meta-data/local-hostname)
public_ip=$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4)
private_ip=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)

# Check disk space usage for root directory
disk_usage=$(df -h / | grep -v Filesystem | awk '{print $5}' | sed 's/%//')

if &#91; "$disk_usage" -ge "$THRESHOLD" ]; then
  message="Disk usage on EC2 instance '${hostname}' is at ${disk_usage}%.
  \nPrivate IP: ${private_ip}
  \nPublic IP: ${public_ip}.
  \nPlease extend disk space!"
  
  payload="{'text': '$message'}"
  curl -X POST -H 'Content-type: application/json' --data "$payload" $SLACK_WEBHOOK_URL
fi

Slack Webhook Setup

In your Slack workspace, navigate to the Slack API page and create an incoming webhook for the desired channel.

Replace SLACK_WEBHOOK_URL in the script with the generated webhook URL.

Pros and Cons of This Approach

Pros:

Cost-Effective:There is no need for additional AWS services like CloudWatch, which can incur costs for custom metrics. The only cost involved is the EC2 instance itself and the minimal bandwidth for sending Slack notifications.

Simple and Lightweight:The solution uses native tools (Bash, curl, cron) available on almost all Linux-based systems. There’s no need to install additional monitoring agents.

Immediate Notifications:Using Slack as the notification channel ensures that your team receives alerts in real-time without needing to log in to AWS or set up more complex monitoring dashboards.

Customizable:You can easily customize the script to check different thresholds, monitor additional directories, or modify the frequency of checks.

Low Overhead:Running a lightweight cron job has minimal impact on the system's performance. This is especially useful for smaller EC2 instances.

Cons:

No Centralized Monitoring:Since the script runs on individual EC2 instances, there’s no centralized dashboard for viewing the disk usage across all instances. You only get notified when a threshold is breached, not before.

Maintenance:If you have many EC2 instances, you’ll need to deploy and maintain this script across all of them. In contrast, centralized solutions like AWS CloudWatch provide an aggregated view with minimal per-instance configuration.

Limited to Disk Monitoring:This script only monitors disk usage, whereas services like CloudWatch can monitor multiple aspects of EC2 health, such as CPU, memory, and network traffic.

Requires Slack Integration:If your organization doesn’t use Slack, you’ll need to integrate the script with another messaging platform, or use email for notifications.

Conclusion

This Bash script offers a simple, low-cost solution to monitor EC2 disk space and send alerts to Slack when disk usage exceeds a critical threshold. It’s an excellent fit for organizations or projects that don’t need the complexity and costs associated with AWS CloudWatch or other comprehensive monitoring tools.

However, it’s important to weigh the simplicity and cost savings against the limitations of this approach, especially when it comes to centralized management and broader system health monitoring. For small to mid-size deployments or cost-conscious environments, this can be a highly effective method to ensure your EC2 instances are not running out of disk space.

Would you like to give this solution a try or explore more comprehensive alternatives? Let me know in the comments!

Let me know if you need any more changes or additions to this article!

---

# Securing Docker Containers: Best Practices for Container Security

URL: https://ercan.cloud/securing-docker-containers-best-practices-for-container-security/
Date: 2024-09-20T15:19:47+03:00

When we talk about containerization, Docker is often the first tool that comes to mind. It's revolutionized how we develop, ship, and ...


When we talk about containerization, Docker is often the first tool that comes to mind. It's revolutionized how we develop, ship, and deploy applications. But with great power comes great responsibility, right? As much as Docker can streamline processes, security should always be top of mind. A vulnerable container can put your whole system at risk.

So, how do you secure your Docker containers? Let’s break it down with some best practices!

1. Keep Your Docker Engine Updated

It sounds basic, but you'd be surprised how often this is overlooked. Docker regularly releases updates, patching vulnerabilities and introducing security features. Running outdated versions of Docker leaves you exposed to known exploits.

Tip: Set up automatic notifications or version checks to stay on top of new releases.

2. Minimize the Attack Surface with Lightweight Base Images

The bigger the image, the bigger the risk. Large images with unnecessary tools increase your attack surface. Stick with minimal base images like alpine or scratch, and only add what’s necessary for your application to run.

Example:

FROM alpine:3.12

This small, lightweight image minimizes unnecessary packages, reducing vulnerabilities.

3. Use Multi-Stage Builds

Multi-stage builds allow you to separate the build environment from the final image, meaning you only include what’s essential in your final Docker image.

Here’s a quick example:

Stage 1 - Build the app

FROM golang:alpine as builder
WORKDIR /app
COPY . .
RUN go build -o myapp

Stage 2 - Use a lightweight image for the final stage

FROM scratch
COPY --from=builder /app/myapp /app/myapp
ENTRYPOINT ["/app/myapp"]

Notice how the second stage is using scratch? This creates a smaller, more secure image by excluding unnecessary build dependencies.

4. Set User Permissions

By default, Docker containers run as root. Big no-no! Running as root gives an attacker unnecessary privileges if they manage to exploit your container. Always run your container processes as a non-root user.

Example:

# Add user and switch to it
RUN useradd -m nonrootuser
USER nonrootuser

This simple tweak can prevent privilege escalation attacks.

5. Limit Container Capabilities

Containers don’t need full privileges to do their job. Docker has Linux capabilities by default, but you can strip them down to the bare minimum. Use the --cap-drop flag to drop unnecessary privileges.

Example:

docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE myapp

Here, we drop all capabilities except for NET_BIND_SERVICE, which is necessary to bind to network ports below 1024.

6. Scan Images for Vulnerabilities

Docker Hub images are great, but they’re not always secure. Vulnerabilities can sneak in. Regularly scan your images using tools like:

Clair (by CoreOS)

Anchore

Trivy (from Aqua Security)

Trivy is an excellent, lightweight tool for scanning images. You can integrate it into your CI/CD pipeline to ensure every image is secure.

Example:

trivy image myapp:latest

7. Enable Docker Content Trust (DCT)

Docker Content Trust (DCT) ensures the integrity and authenticity of the images you pull. It uses digital signatures to verify the source of the image. With DCT enabled, you can’t pull unsigned or tampered images.

To enable Docker Content Trust, simply set the environment variable:

export DOCKER_CONTENT_TRUST=1

8. Use Read-Only File Systems

If your application doesn’t need to write to the filesystem, make it read-only! This reduces the ability for an attacker to modify files if they gain access to the container.

Example:

docker run --read-only myapp

9. Limit Network Exposure

By default, containers are networked and reachable. But do they need to be? Avoid publishing unnecessary ports, and consider using Docker’s network namespaces or private Docker networks for internal communications.

Example:

docker run -d --network my-priv-network myapp

In this case, you’re isolating your container to a private network, limiting exposure to external threats.

10. Enable Logging and Monitoring

Log everything! Tools like Falco, Prometheus, and Grafana can help you monitor container activity and detect unusual behavior in real-time.

Tip: Combine monitoring with alerting. It’s not enough to log – you need to know when something suspicious happens.

11. Use Secrets Management Tools

Storing sensitive data (like API keys or passwords) directly in your container is a bad practice. Use secrets management tools like Docker Secrets, Vault, or AWS Secrets Manager to handle secrets securely.

Example (Docker Secrets):

echo "my-secret-password" | docker secret create db_password -

Then, in your Docker Compose file:

version: "3.1"
services:
  db:
    image: mysql
    secrets:
      - db_password
secrets:
  db_password:
    external: true

12. Regularly Update Your Dependencies

It’s not just your base image that needs updates – don’t forget about your application’s dependencies. Regularly update and patch your libraries and tools to avoid being compromised by known vulnerabilities.

Tip: Use Dependabot or similar tools to automatically check for updates in your dependencies.

Wrapping Up

Securing Docker containers isn’t just a “set it and forget it” task. It’s an ongoing process that requires continuous monitoring, regular updates, and a security-first mindset. By following these best practices, you can dramatically reduce the risk of vulnerabilities while taking advantage of all the benefits Docker has to offer.

The security landscape is constantly evolving, and keeping up with the latest recommendations and tools is key to staying protected. If you’ve got any more tips or questions about container security, feel free to share, we’re all in this containerized world together! 🚀

---

# Mastering Dockerfile: Writing Efficient, Scalable Container Builds

URL: https://ercan.cloud/mastering-dockerfile-writing-efficient-scalable-container-builds/
Date: 2024-09-11T08:04:25+03:00

Docker has revolutionized the way we develop, package, and ship applications. At the heart of this container magic is the Dockerfile, the ...


Docker has revolutionized the way we develop, package, and ship applications. At the heart of this container magic is the Dockerfile, the blueprint for building Docker images. If you want to master Docker, you need to know how to write efficient and scalable Dockerfiles. Let's dive deep into the best practices for crafting a Dockerfile that ensures optimized, lean, and maintainable container images.

1. Start with the Right Base Image

Your choice of base image sets the foundation for your container. The smaller the base, the lighter your resulting image will be. You have two main strategies here:

Alpine Linux: A super lightweight image (~5 MB). It's perfect for microservices or applications where minimal size is critical. However, note that Alpine uses musl instead of glibc, so some libraries may require tweaking.

Official Language Images: For language-specific applications (Node.js, Python, Golang), official images are often preconfigured with necessary dependencies, saving you time. Opt for slim versions (like python:3.9-slim) to avoid bloated images.

Example:

FROM python:3.9-slim

2. Use Multi-stage Builds

Multi-stage builds are a game-changer for keeping Docker images lightweight and clean. You use multiple FROM statements in your Dockerfile, where the first few stages build the application, and the final stage only copies the necessary artifacts.

Why Multi-stage? This approach eliminates build-time dependencies, reducing your final image size dramatically.

Example:

# Build stage
FROM golang:1.17 as builder
WORKDIR /app
COPY . .
RUN go build -o myapp

# Production stage
FROM alpine:latest
WORKDIR /app
COPY --from=builder /app/myapp .
CMD &#91;"./myapp"]

In this example, the final image only contains the compiled Go binary and the Alpine base image, no Go SDK, no source files, just the essentials!

3. Leverage .dockerignoreJust like .gitignore, .dockerignore helps you exclude unnecessary files and directories from being added to your image. For example, why include your .git directory, local environment configs, or temporary files in your image?

This is an easy win for both performance and security.

Example .dockerignore file:

.git
node_modules
*.log
.env

4. Minimize Layer Creation

Each line in a Dockerfile creates a new layer. Docker caches these layers, but too many layers can lead to performance hits and larger image sizes. Try to group related commands into a single RUN instruction.

Before:

RUN apt-get update
RUN apt-get install -y python3
RUN pip3 install -r requirements.txt

After:

RUN apt-get update && apt-get install -y python3 && pip3 install -r requirements.txt

This approach reduces the number of layers and results in a more compact image.

5. Use COPY Instead of ADD

Both COPY and ADD let you move files into the image, but COPY is generally preferred because it’s more explicit and simpler. ADD has extra features like unpacking tarballs and fetching URLs, but this adds complexity. If you don't need those extras, stick with COPY.

Example:

COPY ./source /app/source

6. Use Minimal User Permissions

By default, Docker containers run as the root user, which can be a security risk. Always aim to run your applications as a non-root user.

Example:

# Create a non-root user
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
USER appuser

7. Optimize Caching

Docker caches layers to speed up subsequent builds. Structure your Dockerfile in a way that takes full advantage of this caching. Put frequently-changing layers (like source code) at the bottom and more static layers (like dependencies) at the top.

Example:

# Install dependencies first (less frequent changes)
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy application code (more frequent changes)
COPY . .

With this approach, Docker only re-builds the later layers when code changes, speeding up the build process.

8. Clean Up After Yourself

Commands like apt-get install can leave behind unnecessary files like caches and package lists. Clean them up to reduce image size.

Example:

RUN apt-get update && apt-get install -y \
    python3 \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

This keeps your image lean and avoids leftover cruft.

9. Label Your Images

Adding metadata using LABEL makes your Docker images more manageable, especially in large environments. Use labels for versioning, authorship, and other useful metadata.

Example:

LABEL maintainer="yourname@company.com"
LABEL version="1.0.0"
LABEL description="My Python App"

10. Test Locally Before Pushing

Always test your Docker builds locally before pushing them to a registry or deploying to production. Use docker build and docker run to verify that your image is working as expected.

docker build -t myapp .
docker run --rm -p 8080:8080 myapp

Final Thoughts:

Writing a Dockerfile that’s both efficient and scalable doesn’t have to be daunting. By focusing on best practices like using multi-stage builds, minimizing layers, and optimizing caching, you can build images that are not only lightweight but also fast to deploy and easy to maintain. As you continue to build Dockerfiles, you’ll develop your own strategies for balancing image size, build speed, and functionality.

Remember: the goal is to create Docker images that scale with your application while maintaining performance and security. Keep experimenting, keep optimizing, and most importantly, keep shipping!

That’s it! Now, it’s time to go and try these practices out in your next Docker project. Let me know how it goes, or if you have any more tips to add. Happy containerizing! 🚀

---

# Migrating a Git Repository from GitLab to GitHub with GPG-Signed Commits

URL: https://ercan.cloud/migrating-a-git-repository-from-gitlab-to-github-with-gpg-signed-commits/
Date: 2024-09-02T05:34:58+03:00

In today's DevOps-driven world, version control systems like Git are the backbone of software development. GitLab and GitHub are two of ...


Here’s a comprehensive guide on Migrating a Git Repository from GitLab to GitHub with GPG-Signed Commits:

Migrating a Git Repository from GitLab to GitHub with GPG-Signed Commits

Introduction

In today's DevOps-driven world, version control systems like Git are the backbone of software development. GitLab and GitHub are two of the most popular platforms for managing Git repositories. Developers and teams may choose to migrate their repositories from GitLab to GitHub for various reasons, including taking advantage of GitHub's extensive integrations, community, and feature set.

A key aspect of maintaining the integrity and authenticity of your code when migrating repositories is to ensure that all commits are GPG-signed. GPG signatures help verify the identity of the commit author and prevent tampering with commit history. This guide will provide a step-by-step process to migrate a GitLab repository to GitHub while re-signing all commits with a new GPG key for GitHub.

Why Migrate from GitLab to GitHub?

Both GitLab and GitHub offer robust features, but there are several reasons why a team or developer might want to migrate:

Wider Ecosystem: GitHub has a larger community, extensive third-party integrations, and strong CI/CD workflows with GitHub Actions.

GitHub Sponsors and Discussions: If you want to monetize your projects or need a platform for community discussions, GitHub offers these unique features.

Project Exposure: With GitHub being more popular, hosting your project there can potentially increase visibility.

Personal Preference or Organizational Mandates: Sometimes, a migration is needed due to policy changes or personal comfort with a platform.

Regardless of the reason, the migration should be smooth and should preserve all critical information, including commit signatures.

Understanding GPG-Signed Commits

Before diving into the migration process, it’s important to understand what GPG-signed commits are and why they matter. GPG (GNU Privacy Guard) is a tool for secure communication and data storage. When used in Git, GPG signing ensures that commits are verifiable and that the person committing the code is the actual author. This helps maintain the integrity of the repository, especially in collaborative environments.

GitHub and GitLab both support GPG-signed commits. However, each platform requires that the GPG key used for signing is associated with a specific user account. This means when migrating repositories, you’ll need to re-sign commits with the GPG key tied to your GitHub account.

Step-by-Step Guide to Migrating GitLab to GitHub with GPG-Signed Commits

Prerequisites

Before starting the migration, make sure you have:

Access to the GitLab repository you want to migrate.

Created a new repository on GitHub.

Git installed on your local machine.

A GPG key set up and added to your GitHub account. If you haven’t set this up yet, refer to GitHub’s documentation on GPG keys to create and associate a GPG key with your GitHub account.

Step 1: Cloning the GitLab Repository Locally

The first step is to clone the GitLab repository to your local machine. Since you will be re-signing the commits, it’s best to clone it as a bare repository:

git clone --bare https://gitlab.com/username/repository.git
cd repository.git

The --bare option allows you to clone the repository without a working directory. This is particularly useful for mirroring or migrating repositories.

Step 2: Create a New Repository on GitHub

Go to your GitHub account and create a new repository. Make sure to name it appropriately to reflect the purpose of the project. Take note of the HTTPS or SSH URL of your new repository, as you'll need it for the next step.

Step 3: Add GitHub as a Remote Repository

Once your GitHub repository is ready, add it as a remote to your locally cloned repository:

git remote add github https://github.com/username/new-repository.git

Replace username and new-repository with your GitHub username and the name of your new repository.

Step 4: Re-sign All Commits with Your GitHub GPG Key

Now comes the critical step: re-signing all commits with the GPG key associated with your GitHub account. For this, you can use git filter-repo, a powerful tool for rewriting Git repository history.

Installing git filter-repo

If you haven't already installed git filter-repo, you can do so using Python's pip:

pip install git-filter-repo

Configuring Git with Your GitHub GPG Key

Set up your local Git environment to use your GitHub GPG key for signing:

git config user.name "Your GitHub Name"
git config user.email "your_github_email@example.com"
git config user.signingkey "your_gpg_key_id"

Replace your_github_email@example.com and your_gpg_key_id with your actual GitHub email and GPG key ID.

Re-sign Commits Using git filter-repo

Now, use git filter-repo to re-sign all the commits:

git filter-repo --commit-callback '
commit.author_name = b"Your GitHub Name"
commit.author_email = b"your_github_email@example.com"
commit.committer_name = b"Your GitHub Name"
commit.committer_email = b"your_github_email@example.com"
commit.committer_date = commit.committer_date
' --tag-rename '' --commit-signature ""

# Re-sign all commits with the GPG key
git log --reverse --format=%H | while read commit; do
    GIT_COMMITTER_DATE="$(git show -s --format=%cD "$commit")" git commit --amend --no-edit --date "$(git show -s --format=%cD "$commit")" --gpg-sign="your_gpg_key_id"
done

This script will re-sign all commits in the repository with your GitHub GPG key.

Step 5: Push the Re-signed Repository to GitHub

Now that all the commits are re-signed, push the repository to GitHub:

git push --force --tags github main

The --force option is necessary because the commit history has been rewritten.

Step 6: Verify GPG Signatures on GitHub

To ensure the migration was successful, navigate to your GitHub repository and check the commit history. Each commit should display a "Verified" badge, confirming that it was signed with your GPG key.

Step 7: Clean Up Local Repository (Optional)

After you’ve verified that everything is correctly set up on GitHub, you can remove the local bare repository if you no longer need it:

cd ..
rm -rf repository.git

Best Practices for Repository Migration

Migrating repositories between platforms can be tricky, especially when dealing with signed commits and maintaining commit history integrity. Here are some best practices:

Backup Your Repository: Before making any changes or starting the migration process, create a backup of your repository.

Test the Migration on a Sample Repository: If you're new to repository migration or working with GPG-signed commits, consider testing the process on a sample repository first.

Maintain Clear Documentation: Document every step of the migration process, including configurations and commands used, for future reference or team members.

Regularly Check Repository Integrity: After migration, periodically verify the integrity of the repository and GPG signatures to ensure everything remains intact.

Conclusion

Migrating a Git repository from GitLab to GitHub while preserving GPG-signed commits requires careful attention to detail. This guide walks you through each step, from cloning the GitLab repository to re-signing all commits with a new GPG key and pushing the repository to GitHub. By following this process, you can ensure the integrity and authenticity of your commit history are maintained, giving your team and collaborators confidence in the migrated repository.

This approach not only helps maintain a clean history but also leverages the security benefits of GPG-signed commits on GitHub. As more teams and developers recognize the importance of commit verification, mastering these steps becomes an essential skill in your DevOps toolkit.

With the information provided in this guide, you should be well-equipped to perform a smooth and secure migration from GitLab to GitHub, maintaining GPG-signed commit integrity throughout the process.

FAQs

Q1: What if I don’t have a GPG key associated with my GitHub account?
A1: You can easily generate a new GPG key and add it to your GitHub account by following the GitHub GPG setup guide.

Q2: Can I automate this process for multiple repositories?
A2: Yes, you can write a script to automate the cloning, re-signing, and pushing processes for multiple repositories. However, testing is crucial to avoid any data loss or corruption.

Q3: Are there any limitations when using git filter-repo?
A3: git filter-repo is a powerful tool, but it’s important to thoroughly read its documentation and understand its commands before using it on critical repositories.

Q4: What if I encounter errors while pushing to GitHub?
A4: Ensure that your GitHub repository is set up correctly and that you have sufficient permissions. Additionally, check that your network and

authentication configurations are correct.

By following the steps in this guide, you can confidently migrate your GitLab repositories to GitHub while maintaining the integrity of your commit history with GPG-signed commits.

This article should provide you with all the details needed for a successful migration and should be SEO-friendly for a technical audience looking for a comprehensive guide on this topic.

---

# Accessing AWS Services in Private Subnets Without 0.0.0.0/0

URL: https://ercan.cloud/accessing-aws-services-in-private-subnets-without-0-0-0-0-0/
Date: 2024-08-24T14:56:44+03:00

When working with AWS (Amazon Web Services), securing your infrastructure is paramount. One of the most common security practices is to ...


When working with AWS (Amazon Web Services), securing your infrastructure is paramount. One of the most common security practices is to restrict access to your private subnets by avoiding the use of 0.0.0.0/0, which represents all IP addresses globally. While this enhances security, it can also present challenges when your applications and services need to interact with various AWS services. This article will guide you on how to securely access AWS services from private subnets without exposing your resources to the public internet.

In this guide, we’ll explore different methods to achieve this, dive into specific AWS services, and provide practical examples to help you implement these strategies. Whether you're a seasoned AWS user or just starting out, this guide will equip you with the knowledge to enhance the security and functionality of your cloud infrastructure.

Why Avoid 0.0.0.0/0?

The IP address 0.0.0.0/0 is shorthand for "anywhere." When used in security groups or network access controls, it allows access from any IP address in the world. While this might be necessary in some scenarios, it’s generally considered risky because it exposes your resources to potential attacks. By restricting access to specific IP ranges, you can significantly reduce the attack surface of your infrastructure.

Strategies to Access AWS Services Without 0.0.0.0/0

1. Using VPC Endpoints

AWS VPC (Virtual Private Cloud) Endpoints allow you to privately connect your VPC to supported AWS services without requiring an Internet Gateway, NAT device, VPN connection, or AWS Direct Connect. Traffic between your VPC and AWS services does not leave the Amazon network, providing enhanced security and performance.

There are two types of VPC Endpoints:

Interface Endpoints: These are powered by AWS PrivateLink and are used to access services such as EC2, KMS, CloudWatch, and more. They create an elastic network interface in your subnet with a private IP address, enabling communication with the AWS service.

Gateway Endpoints: These are used to access Amazon S3 and DynamoDB. They act as a target for a route in your route table, directing traffic for the specified service to the endpoint.

Example Services:

Amazon S3

Amazon DynamoDB

Amazon EC2

AWS Systems Manager

Amazon SNS

Amazon SQS

2. NAT Gateways

A NAT (Network Address Translation) Gateway is a managed service that allows instances in a private subnet to connect to the internet or other AWS services, but prevents the internet from initiating a connection with those instances. Unlike an Internet Gateway, a NAT Gateway does not allow inbound traffic from the internet, making it a safer option for accessing public AWS services from a private subnet.

NAT Gateways are highly available within a single Availability Zone and can be scaled to meet your traffic demands. When deploying a NAT Gateway, it’s recommended to set it up in a public subnet and configure the route tables of your private subnets to use the NAT Gateway for internet-bound traffic.

Example Services:

Amazon EC2

Amazon RDS

Amazon Redshift

AWS Lambda

Amazon ElastiCache

3. AWS Transit Gateway

AWS Transit Gateway enables you to connect your VPCs, on-premises networks, and AWS services through a central hub. It acts as a highly scalable cloud router that simplifies network architecture and ensures secure, scalable, and managed connections between your resources.

Transit Gateway allows you to route traffic between your VPCs and on-premises networks without needing a direct internet connection. This is particularly useful for large-scale architectures where multiple VPCs or regions need to interact securely.

Example Services:

Amazon VPC

AWS Direct Connect

AWS Site-to-Site VPN

Amazon S3 (via VPC Endpoints)

Amazon DynamoDB (via VPC Endpoints)

4. AWS Direct Connect

AWS Direct Connect provides a dedicated network connection from your premises to AWS. It bypasses the internet entirely, offering a more secure and reliable way to connect your private subnets to AWS services. Direct Connect is often used in hybrid cloud architectures where on-premises resources need to interact with AWS services securely.

By using Direct Connect, you can route traffic from your private subnet to AWS services via a private connection, ensuring that your data remains within the AWS network and reducing exposure to external threats.

Example Services:

Amazon EC2

Amazon RDS

AWS Lambda

Amazon S3

Amazon CloudFront

5. AWS PrivateLink

AWS PrivateLink allows you to securely access services hosted on AWS in a private and highly available manner. It eliminates the need for a NAT device, Internet Gateway, or VPN connection. PrivateLink establishes a direct connection between your VPC and the AWS service through an interface endpoint.

This service is particularly useful for accessing third-party SaaS applications or custom services hosted within AWS. With PrivateLink, traffic remains within the AWS network, further securing your interactions with external services.

Example Services:

Amazon CloudWatch

AWS Secrets Manager

AWS CodeBuild

Amazon ECR

Amazon Kinesis

6. VPC Peering

VPC Peering allows you to connect one VPC with another, enabling instances in either VPC to communicate as if they were within the same network. VPC Peering is a simple, one-to-one connection between VPCs, which can be within the same AWS account or different accounts.

While VPC Peering does not inherently provide access to AWS services, it can be combined with other methods (such as VPC Endpoints) to allow instances in a private subnet to securely access services hosted in another VPC.

Example Services:

Amazon EC2

Amazon RDS

Amazon S3 (via VPC Endpoints)

Amazon DynamoDB (via VPC Endpoints)

Amazon EKS

7. Elastic Load Balancers (ELB)

Elastic Load Balancers (ELB) distribute incoming traffic across multiple targets, such as EC2 instances, containers, or IP addresses, in one or more Availability Zones. By placing an ELB in a private subnet and using it to route traffic to your resources, you can securely expose your services to other AWS services without direct internet exposure.

For example, an Application Load Balancer (ALB) can route traffic from an interface endpoint to your EC2 instances, while a Network Load Balancer (NLB) can manage traffic from your on-premises resources to AWS services.

Example Services:

Amazon EC2

Amazon ECS

AWS Fargate

AWS Lambda

Amazon RDS

8. AWS Systems Manager (SSM)

AWS Systems Manager provides a secure, scalable way to manage your AWS resources. With Systems Manager, you can remotely access and manage your EC2 instances and other resources in private subnets without requiring a direct internet connection.

SSM Session Manager, a feature of Systems Manager, allows you to securely connect to your instances without the need for SSH or RDP access, further reducing the need for 0.0.0.0/0. It also integrates with IAM, providing fine-grained access control.

Example Services:

Amazon EC2

Amazon RDS

AWS Lambda

Amazon CloudWatch

AWS Config

AWS Services and Their Private Access Solutions

Let’s take a closer look at how you can securely access various AWS services from private subnets using the strategies mentioned above:

1. Amazon S3

Access Method: Gateway VPC Endpoint

Description: You can create a Gateway VPC Endpoint for Amazon S3, allowing your instances in private subnets to access S3 buckets without exposing your resources to the internet.

2. Amazon DynamoDB

Access Method: Gateway VPC Endpoint

Description: Similar to S3, DynamoDB can be accessed securely from private subnets by creating a Gateway VPC Endpoint.

3. Amazon EC2

Access Method: NAT Gateway, VPC Peering, SSM

Description: EC2 instances in private subnets can access public AWS services through a NAT Gateway or connect with other VPCs via VPC Peering.

4. Amazon RDS

Access Method: NAT Gateway, VPC Peering

Description: You can access your RDS instances from private subnets using a NAT Gateway or VPC Peering.

5. Amazon Redshift

Access Method: NAT Gateway, Interface VPC Endpoint

Description: Redshift clusters can be accessed via NAT Gateways or Interface VPC Endpoints, ensuring private subnet resources can interact securely.

6. AWS Lambda

Access Method: VPC Endpoints, SSM

Description: Lambda functions can access private subnets via VPC Endpoints, and you can manage them securely with AWS Systems Manager.

7. Amazon ElastiCache

Access Method: NAT Gateway, VPC Peering

Description: ElastiCache can be accessed from private subnets using NAT Gateways or VPC Peering.

8. Amazon CloudWatch

Access Method: Interface VPC Endpoint, SSM

Description: Monitor your resources in private subnets by accessing CloudWatch through an Interface VPC Endpoint or managing them with SSM.

9. AWS Secrets Manager

Access Method: Interface VPC Endpoint, PrivateLink

Description: Securely store and retrieve secrets from your private subnets using an Interface VPC Endpoint or PrivateLink.

10. Amazon ECR

Access Method: Interface VPC Endpoint, NAT Gateway

Description: Access container images stored in ECR from private subnets using Interface VPC Endpoints or NAT Gateways.

11. Amazon SQS

Access Method: Interface VPC Endpoint, NAT Gateway

Description: Securely send and receive messages from SQS using Interface VPC Endpoints or NAT Gateways.

12. Amazon SNS

Access Method: Interface VPC Endpoint, NAT Gateway

Description: Send notifications to SNS from private subnets through Interface VPC Endpoints or NAT Gateways.

13. Amazon Kinesis

Access Method: Interface VPC Endpoint, PrivateLink

Description: Securely process streaming data using Kinesis in private subnets via Interface VPC Endpoints or PrivateLink.

14. AWS CodeBuild

Access Method: Interface VPC Endpoint, PrivateLink

Description: Run your build processes in private subnets by accessing CodeBuild with Interface VPC Endpoints or PrivateLink.

15. AWS CloudFormation

Access Method: Interface VPC Endpoint, NAT Gateway

Description: Deploy and manage your infrastructure using CloudFormation in private subnets via Interface VPC Endpoints or NAT Gateways.

16. AWS Step Functions

Access Method: Interface VPC Endpoint, NAT Gateway

Description: Orchestrate workflows in private subnets by accessing Step Functions through Interface VPC Endpoints or NAT Gateways.

17. Amazon EFS

Access Method: VPC Peering, PrivateLink

Description: Share files between private subnets securely using EFS with VPC Peering or PrivateLink.

Conclusion

Securing access to AWS services from private subnets is essential for maintaining the security and integrity of your cloud environment. By avoiding the use of 0.0.0.0/0 and leveraging the various AWS networking features like VPC Endpoints, NAT Gateways, and AWS Direct Connect, you can ensure that your infrastructure remains secure while still being able to interact with necessary AWS services.

Understanding and implementing these strategies will not only enhance the security of your AWS environment but also improve performance and reliability. With the knowledge gained from this guide, you can confidently build and manage a robust cloud architecture that meets the highest security standards.

By integrating these methods into your architecture, you’ll be well on your way to creating a secure, efficient, and scalable AWS environment that minimizes exposure to the public internet while maximizing functionality.

For more detailed instructions on implementing these methods, refer to the official AWS documentation or consult with an AWS Solutions Architect to tailor the solutions to your specific needs.

---

# Understanding AWS Regions, Availability Zones, and VPCs: A Comprehensive Guide

URL: https://ercan.cloud/understanding-aws-regions-availability-zones-and-vpcs/
Date: 2024-08-21T19:17:28+03:00

In today's cloud computing landscape, Amazon Web Services (AWS) has established itself as a leader, offering a wide range of services to ...


Introduction

In today's cloud computing landscape, Amazon Web Services (AWS) has established itself as a leader, offering a wide range of services to organizations of all sizes. Among its core features are AWS Regions, Availability Zones (AZs), and Virtual Private Clouds (VPCs), which form the foundation of the AWS infrastructure. Understanding these concepts is crucial for anyone looking to architect scalable, reliable, and secure cloud solutions.

In this comprehensive guide, we'll delve into the intricacies of AWS Regions, Availability Zones, and VPCs. Whether you're a cloud engineer, DevOps professional, or just starting out with AWS, this post will equip you with the knowledge you need to make informed decisions when designing your cloud architecture.

What are AWS Regions?

Definition and Overview

AWS Regions are geographic locations where AWS has data centers. Each region consists of multiple isolated and physically separated data centers known as Availability Zones. AWS currently offers over 30 regions worldwide, with more planned for the future. These regions allow you to choose the physical location where your data is stored and where your applications run, which can be critical for compliance, latency, and disaster recovery planning.

Why Regions Matter

Choosing the right region is critical for several reasons:

Compliance and Data Sovereignty: Certain industries or regions have regulations that require data to be stored within specific geographic boundaries. AWS Regions allow you to comply with these regulations by selecting a region within the required jurisdiction.

Latency: The closer your users are to the region where your application is hosted, the lower the latency. This can significantly improve the user experience, especially for latency-sensitive applications.

Cost: Pricing varies between regions. Some regions are more expensive due to factors like energy costs, local taxes, and demand. It's essential to consider the cost implications when selecting a region.

How to Choose the Right Region

When choosing an AWS Region, consider the following factors:

Geographical proximity to users: Choose a region close to your primary user base to reduce latency.

Compliance requirements: Ensure the region meets any regulatory requirements for data storage and processing.

Service availability: Not all AWS services are available in every region. Make sure the services you need are available in the region you're considering.

Cost: Compare the pricing for the services you plan to use across different regions.

Availability Zones: The Building Blocks of AWS Reliability

What are Availability Zones?

Availability Zones (AZs) are isolated data centers within a region. Each AZ is a fully independent facility with its own power, cooling, and networking. AZs within a region are interconnected through low-latency links, allowing for high availability and fault tolerance.

Importance of Availability Zones

Availability Zones play a crucial role in ensuring the reliability and resilience of your applications:

High Availability: By deploying resources across multiple AZs, you can design applications that remain operational even if one AZ experiences a failure. This redundancy is essential for mission-critical applications.

Fault Tolerance: AZs are physically separated, reducing the risk of simultaneous failure. This separation means that an issue in one AZ, such as a power outage or hardware failure, won't impact other AZs.

Disaster Recovery: Multi-AZ deployments provide a robust disaster recovery strategy. In the event of a disaster in one AZ, your application can continue to function in another AZ with minimal disruption.

Best Practices for Using Availability Zones

To maximize the benefits of Availability Zones, follow these best practices:

Distribute resources: Always deploy your resources across multiple AZs to ensure high availability and fault tolerance.

Leverage load balancers: Use AWS services like Elastic Load Balancing (ELB) to distribute traffic across AZs, ensuring seamless failover in case of an AZ outage.

Utilize multi-AZ databases: For critical applications, consider using multi-AZ deployments for databases like Amazon RDS to ensure data redundancy and availability.

Understanding VPCs: Your Private Cloud within AWS

What is a VPC?

A Virtual Private Cloud (VPC) is a logically isolated network within the AWS cloud. It allows you to launch AWS resources, such as EC2 instances, RDS databases, and Lambda functions, in a virtual network that you define. With a VPC, you have complete control over your network settings, including IP address ranges, subnets, route tables, and security groups.

Key Components of a VPC

Subnets: A subnet is a range of IP addresses within a VPC. Subnets can be public or private, depending on their routing to the internet. Public subnets have a route to an internet gateway, allowing resources to be accessible from the internet, while private subnets do not, keeping resources isolated from direct internet access.

Route Tables: A route table contains a set of rules, called routes, that determine where network traffic is directed. Each subnet must be associated with a route table, which defines the routing for that subnet.

Internet Gateway (IGW): An IGW allows communication between resources in your VPC and the internet. It is required for any public subnet in your VPC.

NAT Gateway/Instance: A Network Address Translation (NAT) gateway or instance enables instances in a private subnet to initiate outbound traffic to the internet while preventing inbound traffic.

Security Groups and Network ACLs: Security groups act as virtual firewalls for your instances, controlling inbound and outbound traffic at the instance level. Network ACLs (Access Control Lists) are another layer of security that acts at the subnet level.

Benefits of Using a VPC

Using a VPC provides several advantages:

Isolation and Security: A VPC offers a high level of isolation, allowing you to create secure environments for your applications. With fine-grained control over network traffic, you can enforce strict security policies.

Customization: VPCs are highly customizable, giving you control over your network configuration. You can design your VPC to meet specific requirements, such as hybrid cloud architectures or multi-tier applications.

Scalability: VPCs can scale with your needs, allowing you to add or remove resources as your application grows.

Cost-Effective: By optimizing your VPC design, you can reduce costs. For example, placing certain resources in private subnets can lower data transfer costs compared to public subnets.

How to Design a VPC for High Availability

Designing a VPC for high availability involves careful planning and adherence to best practices:

Multi-AZ Deployment: Spread your resources across multiple Availability Zones to ensure high availability. This approach minimizes the risk of a single point of failure.

Redundant Internet Gateways: Consider using multiple internet gateways to increase redundancy and availability for public-facing applications.

Automated Failover: Implement automated failover mechanisms, such as using Elastic Load Balancers, to route traffic to healthy instances across AZs.

Best Practices for Architecting AWS Environments with Regions, AZs, and VPCs

Designing for Scalability and Flexibility

To ensure your AWS environment can scale and adapt to changing needs, consider the following best practices:

Use Auto Scaling: AWS Auto Scaling automatically adjusts the number of instances in response to demand, ensuring that your application always has the right amount of resources. Deploying Auto Scaling groups across multiple AZs enhances availability and fault tolerance.

Implement Elastic Load Balancing: ELB distributes incoming traffic across multiple targets, such as EC2 instances, containers, or IP addresses. This distribution improves fault tolerance and ensures consistent application performance.

Optimize VPC Peering: If you have multiple VPCs, consider using VPC peering to allow them to communicate with each other. This can be useful for hybrid cloud architectures or when segmenting environments for security purposes.

Security Considerations

Security should be a top priority when designing your AWS environment:

Use IAM Policies: AWS Identity and Access Management (IAM) allows you to define fine-grained access control policies. Ensure that users and roles have the least privilege necessary to perform their tasks.

Encrypt Data: Use encryption for both data at rest and in transit. AWS services like S3, RDS, and EBS offer built-in encryption options.

Monitor and Audit: Use AWS services like CloudTrail and CloudWatch to monitor and audit activities in your AWS environment. Set up alerts for unusual activities and regularly review logs.

Cost Optimization Strategies

Optimizing costs is an essential aspect of cloud architecture:

Choose the Right Instance Types: Select the appropriate instance types based on your workload requirements. AWS offers a wide range of instance types, each optimized for different use cases.

Use Reserved Instances: If you have steady-state workloads, consider using Reserved Instances (RIs) to save costs. RIs offer significant discounts compared to on-demand pricing.

Leverage Spot Instances: For flexible workloads, use Spot Instances, which allow you to take advantage of unused EC2 capacity at a lower cost.

Disaster Recovery and Business Continuity

Ensure your AWS environment is resilient and can recover from disasters:

Implement Multi-Region Architectures: For critical applications, consider deploying resources across multiple regions. This approach provides protection against regional failures and ensures business continuity.

Use Backup and Restore Strategies: Regularly back up your data using AWS services like S3, Glacier, and AWS Backup. Ensure that your backup strategy aligns with your Recovery Point Objectives (RPO) and Recovery Time Objectives (RTO).

Test Disaster Recovery Plans: Regularly test your disaster recovery plans to ensure they work as expected. Simulate failures and practice failover to verify that your applications can recover quickly.

Real-World Use Cases

E-commerce Platforms

E-commerce platforms require high availability and low latency to ensure a seamless shopping experience for customers. By deploying their infrastructure across multiple AZs and using VPCs for network isolation, e-commerce companies can achieve the reliability and performance needed to handle peak traffic during sales events.

Financial Services

Financial services organizations must adhere to strict regulatory requirements and ensure data security. Using AWS Regions that comply with local regulations, combined with VPCs for secure network segmentation, helps these organizations maintain compliance while delivering fast, reliable services.

Media and Entertainment

Media and entertainment companies often deal with large volumes of data and require scalable infrastructure to support content delivery. By leveraging AWS Regions and AZs, these companies can distribute content globally, ensuring low-latency access for users worldwide.

Healthcare

Healthcare providers need to store and process sensitive patient data while complying with regulations like HIPAA. AWS Regions and VPCs provide the necessary tools to build secure, compliant environments that protect patient data and ensure continuity of care.

Conclusion

AWS Regions, Availability Zones, and VPCs are fundamental components of the AWS cloud infrastructure. Understanding how to leverage these concepts effectively is crucial for designing scalable, reliable, and secure applications. By following best practices and considering factors like compliance, latency, and cost, you can build robust cloud architectures that meet your organization's needs.

Whether you're deploying a simple web application or managing a complex, multi-region infrastructure, the principles outlined in this guide will help you make informed decisions and optimize your AWS environment for success.

---

# AWS Network Load Balancers with Header Modification Techniques

URL: https://ercan.cloud/aws-network-load-balancers-with-header-modification-techniques/
Date: 2024-06-10T23:25:29+03:00

AWS Network Load Balancers (NLBs) are a powerful tool for distributing incoming application traffic across multiple targets...


AWS Network Load Balancers (NLBs) are a powerful tool for distributing incoming application traffic across multiple targets, such as Amazon EC2 instances, in a single or multiple Availability Zones. They provide high throughput, low latency, and are designed to handle millions of requests per second while maintaining ultra-low latencies. However, one limitation of NLBs is the inability to modify HTTP headers directly. This article explores various techniques to work around this limitation, providing full examples to help you implement these solutions in your own AWS environment.

Introduction to AWS Network Load Balancers

AWS Network Load Balancers operate at the connection level (Layer 4), routing connections to targets based on IP protocol data. NLBs are ideal for applications that require extreme performance and low latency, making them suitable for real-time applications like gaming, financial transactions, or video streaming.

Key Features of NLBs:

Static IP: Each NLB is assigned one static IP per Availability Zone (AZ) it operates in.

Elastic IP: You can associate one or more Elastic IP addresses with your NLB.

TLS Termination: NLBs can offload TLS termination to minimize CPU load on your application servers.

Cross-Zone Load Balancing: Distributes traffic evenly across all targets in all enabled AZs.

Target Health Monitoring: Automatically checks the health of targets and routes traffic only to healthy targets.

Header Modification Needs

Modifying HTTP headers is often essential for a variety of use cases, such as:

Adding security headers (e.g., Content-Security-Policy).

Injecting custom headers for logging or tracking.

Modifying User-Agent headers for backend processing.

Implementing security policies based on headers.

Given the Layer 4 operation of NLBs, direct header modification is not feasible. However, we can achieve this using AWS Lambda, Amazon API Gateway, and Application Load Balancers (ALBs).

Solution 1: Using AWS Lambda and Amazon API Gateway

Step 1: Create an AWS Lambda Function

First, create a Lambda function that will modify the headers of incoming requests.

Go to the AWS Lambda console.

Create a new Lambda function:Choose "Author from scratch".

Enter a function name (e.g., ModifyHeadersFunction).

Choose a runtime (e.g., Python 3.8).

Create an execution role with basic Lambda permissions.

Write the Lambda function code:

def lambda_handler(event, context):
    # Log the incoming event
    print("Received event: " + json.dumps(event, indent=2))

    # Modify the headers
    event&#91;'headers']&#91;'X-Custom-Header'] = 'CustomHeaderValue'

    return {
        'statusCode': 200,
        'headers': {
            'Content-Type': 'application/json'
        },
        'body': json.dumps({
            'message': 'Header modified successfully',
            'input': event
        })
    }

Deploy the function.

Step 2: Create an Amazon API Gateway

Next, create an API Gateway that will route requests to the Lambda function.

Go to the Amazon API Gateway console.

Create a new API:Choose "HTTP API".

Set up a new API.

Integrate the API with the Lambda function:Add an integration by selecting the Lambda function created earlier.

Configure routes and methods (e.g., POST /modify).

Deploy the API and note the endpoint URL.

Step 3: Point the NLB to the API Gateway

Finally, set the API Gateway URL as the target for your NLB.

Go to the EC2 console.

Navigate to Load Balancers and select your NLB.

Configure a target group to point to the API Gateway endpoint.

This setup routes all traffic through the API Gateway, where the Lambda function modifies the headers before forwarding the requests to your backend services.

Solution 2: Using AWS WAF

AWS WAF can be used to inspect and modify requests before they reach your backend services, although this method is more about filtering rather than directly modifying headers.

Create a Web ACL in AWS WAF.

Define rules to inspect headers and take actions (e.g., block, allow, or count requests).

Associate the Web ACL with your NLB.

Example Rule:

IF request.headers&#91;'User-Agent'] matches 'bad-bot'
THEN block request

This setup helps in filtering out malicious requests or modifying requests based on certain conditions.

Summary

While AWS Network Load Balancers provide robust performance and scalability, their Layer 4 nature limits direct header modification. However, by leveraging AWS Lambda with Amazon API Gateway or AWS WAF, you can achieve the desired header modification functionality.

Each solution has its own strengths and is suitable for different use cases:

AWS Lambda and API Gateway: Best for specific and complex header manipulations.

AWS WAF: Ideal for security-focused applications needing inspection and filtering.

By understanding these techniques and their implementations, you can effectively manage and modify HTTP headers in your AWS environment, ensuring that your applications remain secure, efficient, and highly available.

If you have any questions or need further assistance with implementing these solutions, feel free to reach me! Happy building on AWS!

---

# How to call multiple terraform modules in a single terragrunt file

URL: https://ercan.cloud/how-to-call-multiple-terraform-modules-in-a-single-terragrunt-file/
Date: 2024-05-07T20:15:35+03:00

in Terragrunt, you can call multiple Terraform modules from a single Terragrunt configuration file by using the terraform block in combi...


in Terragrunt, you can call multiple Terraform modules from a single Terragrunt configuration file by using the terraform block in combination with child configurations. This is typically done by organizing your Terragrunt configuration into a hierarchy where each module is referenced in its own Terragrunt file, but managed centrally using a parent Terragrunt file.

Here's a basic outline of how you can structure this:

Create a Parent Terragrunt File: This file won't directly deploy any resources but will be used to configure common settings and orchestrate module deployment.

Create Child Terragrunt Files for Each Module: Each module will have its own Terragrunt configuration file that specifies the source of the Terraform module and any necessary inputs.

Example Structure

Here’s an example directory structure:

/terraform-live
|-- terragrunt.hcl  # Parent configuration
|-- network
|   |-- terragrunt.hcl  # Module configuration for network
|-- app
|   |-- terragrunt.hcl  # Module configuration for app

Parent Terragrunt.hcl

# /terraform-live/terragrunt.hcl
remote_state {
  backend = "s3"
  config = {
    bucket         = "my-terraform-state"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "us-west-1"
    encrypt        = true
    dynamodb_table = "my-lock-table"
  }
}

# Include all child configurations
include {
  path = find_in_parent_folders()
}

Child Terragrunt.hcl (e.g., for Network)

# /terraform-live/network/terragrunt.hcl
terraform {
  source = "git::https://example.com/network-module.git?ref=v1.0"
}

# Inputs specific to the network module
inputs = {
  vpc_id = "vpc-123456"
}

Usage

To apply all modules, you would typically navigate to each module directory and run Terragrunt commands, like terragrunt apply. You can also automate this with scripts or CI/CD workflows that iterate through module directories and execute Terragrunt commands.

This setup allows you to maintain a clear separation of concerns, reusable code, and consistency across environments while managing the deployment of multiple modules from a centralized configuration.

---

# Leveraging Git Hooks for Enforcing Commit Message Standards: A Guide for Cross-Platform Development Teams

URL: https://ercan.cloud/leveraging-git-hooks-for-enforcing-commit-message-standards-a-guide-for-cross-platform-development-teams/
Date: 2024-04-03T14:33:40+03:00


In the world of software development, maintaining a clean and navigable commit history is not just a matter of neatness but a cornerstone of efficient teamwork and project management. Commit messages serve as a logbook or diary, providing insights into the why and what of each change. This importance grows in multi-developer projects where tracking changes and understanding their purpose can become complex.

This guide introduces Git hooks as a solution to enforce commit message standards, such as SemVer and Conventional Commits, across various operating systems. We'll explore how to implement these checks on Mac, Linux, and Windows, ensuring that your team's contributions are consistent, regardless of the development environment.

What Are Git Hooks?

Git hooks are scripts that run automatically before or after events in the Git lifecycle, such as commit, push, and merge operations. They are a powerful tool for automating and enforcing project rules and workflows, without the need for manual oversight.

Setting Up Your Environment

The implementation of Git hooks involves creating scripts that are triggered at specific stages of the Git workflow. These scripts can be written in various scripting languages, but for simplicity and cross-platform compatibility, we'll focus on shell scripts for Linux and Mac, and batch scripts or PowerShell for Windows.

For Linux and Mac:

Locate the Hooks Directory: Navigate to .git/hooks in your repository. This directory contains sample scripts that can be modified or replaced.

Create a commit-msg Hook: This hook allows you to inspect the commit message before it is finalized. Rename the commit-msg.sample to commit-msg and edit it to include your checks.

For Windows:

Windows users can utilize Git Bash or Windows Subsystem for Linux (WSL) to run shell scripts, making the process similar to that on Linux and Mac. Alternatively, PowerShell scripts can be used but require a different setup approach.

Enforcing Commit Message Standards

The Commit Message Rule:

SemVer and Conventional Commits: Commit messages should start with a type (feat, fix, etc.), optionally followed by a scope, and a succinct description of the change. For example, feat(database): Add user authentication mechanism.

Minimum Word Count: Commit messages must contain at least 10 words to ensure descriptive clarity.

Implementing the Hook:

#!/bin/sh
message=$(cat $1)
word_count=$(echo $message | wc -w)

if ! echo "$message" | grep -qE "^(feat|fix|docs|style|refactor|perf|test|chore)(\(\S+\))?: .{10,}$"; then
    echo "Commit message does not follow Conventional Commits standards."
    exit 1
fi

if &#91; "$word_count" -lt 10 ]; then
    echo "Commit message must be at least 10 words."
    exit 1
fi

Deployment and Maintenance

Distributing and maintaining these hooks across a team can be challenging. It's crucial to ensure every team member installs the necessary hooks. This can be achieved through:

Documentation: Clearly document the setup process in your project's README.

Automation: Use scripts to automate the setup process for new developers.

Version Control: Store hooks in a dedicated repository or within your project (outside .git) and write a script to symlink or copy them to the .git/hooks directory.

Benefits for Personal Development and Team Efficiency

Enforcing commit message standards through Git hooks offers several advantages:

Improved Codebase Understanding: Descriptive messages provide context, making it easier for new team members to understand the project's history.

Streamlined Workflows: Automation reduces the risk of human error, ensuring that only compliant commits are accepted.

Enhanced Collaboration: Standardized commit messages facilitate clearer communication among team members, making it easier to track changes and understand their impact.

Conclusion

Implementing Git hooks to enforce commit message standards is a powerful strategy for maintaining a high-quality codebase and efficient team collaboration. By following the steps outlined in this guide, you can set up a system that works across different operating systems, ensuring consistency and clarity in your project's commit history. This practice not only aids in personal development by fostering better coding habits but also enhances team dynamics by ensuring clear and informative commit messages.

Remember, the key to successful implementation is not just in setting up the hooks but in maintaining a culture of communication and documentation that supports these standards. With the right approach, Git hooks can become an invaluable part of your development workflow, contributing to the overall success and health of your projects.

---

# Simplifying SSL with Let's Encrypt and CLI.ini: A DevOps Guide

URL: https://ercan.cloud/simplifying-ssl-with-lets-encrypt-and-cli-ini-a-devops-guide/
Date: 2024-03-25T12:29:07+03:00

In the vast expanse of the internet, security is not just a luxury; it's a necessity. For web developers and system administrators, ...


In the vast expanse of the internet, security is not just a luxury; it's a necessity. For web developers and system administrators, ensuring that web traffic is encrypted is a foundational aspect of protecting users' data. Enter Let's Encrypt, a free, automated, and open Certificate Authority (CA) that has revolutionized the way we secure websites. This guide aims to unravel the complexities of using Let's Encrypt, focusing on the power of the cli.ini configuration file, and providing a comprehensive introduction to securing your web servers with HTTPS.

Introduction to Let's Encrypt

Let's Encrypt is a free service that provides digital certificates to enable HTTPS (SSL/TLS) for websites, thereby ensuring secure connections. What sets Let's Encrypt apart is its simplicity and automation. With a few commands, you can obtain and renew certificates without the need to manually generate Certificate Signing Requests (CSRs) or configure your server to serve the right certificate.

The Role of Certbot and CLI.ini

Certbot is the most popular Let's Encrypt client, designed to simplify the process of obtaining and renewing certificates. While Certbot's command-line interface is intuitive, managing multiple certificates or automating the renewal process can become tedious. This is where the cli.ini file becomes invaluable.

The cli.ini file allows you to specify Certbot options in a configuration file, making it easier to automate and standardize certificate management across your infrastructure. With cli.ini, you can pre-define settings for email contact, webroot paths, security policies, and more.

Getting Started with Let's Encrypt and Certbot

Before diving into the specifics of the cli.ini file, let's outline the steps to get started with Let's Encrypt and Certbot:

Installation: The first step is to install Certbot on your server. This varies by operating system and web server, but Certbot's website provides tailored instructions for most environments.

Obtaining a Certificate: After installation, you can obtain a certificate using various plugins like webroot, standalone, or nginx. The choice of plugin depends on your server setup and how you prefer to validate domain ownership.

Automating Renewal: Certbot can automatically renew your certificates before they expire. Setting up a cron job or a systemd timer ensures your certificates are always up to date without manual intervention.

Understanding CLI.ini Configuration

The cli.ini file is the heart of automating and customizing Certbot's behavior. Located in Certbot's configuration directory (mostly /etc/letsencrypt/cli.ini), this file contains global flags and settings applied to every Certbot command. Here's how to make the most of it:

Basic Configuration Options

Email and Agreement: Automate the agreement to terms of service and specify your contact email for important notifications.

email = user@example.com
agree-tos = True

Non-Interactive Mode: Essential for automation, this option runs Certbot without waiting for user input.

non-interactive = True

Security and Performance

Key Size and Type: Define the RSA key size or opt for an ECDSA key for better performance.

rsa-key-size = 2048
key-type = ecdsa
elliptic-curve = secp384r1

OCSP Must-Staple: Enhance security by requesting certificates with the OCSP Must-Staple extension.

must-staple = True

Domain and Webroot Configuration

For those using the webroot plugin, specifying your domains and webroot paths in cli.ini simplifies the command you need to run.

authenticator = webroot
domains = example.com,www.example.com
webroot-path = /var/www/html

Hooks: Automating Service Management

Certbot can execute scripts before and after certificate renewal, allowing you to automate service restarts or configuration changes.

pre-hook = systemctl stop nginx
post-hook = systemctl start nginx

Advanced Configuration: Standalone Mode and HTTP Challenges

If you're using Certbot's standalone mode, especially when no web server is running, you might need to specify alternative ports for the HTTP-01 challenge.

authenticator = standalone
http-01-port = 8080

---

# TCP vs. UDP: Navigating the Protocols as Developers and DevOps

URL: https://ercan.cloud/tcp-vs-udp-navigating-the-protocols-as-developers-and-devops/
Date: 2024-03-05T19:06:53+03:00

Understanding TCP and UDPBefore we dive into the differences, let's briefly understand what TCP and UDP are ...


Today, we embark on a fascinating exploration of two cornerstone protocols of the internet: TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). Understanding the nuances of TCP and UDP is crucial for developers and DevOps professionals alike, as it influences everything from application design to network troubleshooting. So, let's delve deep into the realms of TCP and UDP, shedding light on their differences, use cases, and what you need to know to harness their full potential.

Understanding TCP and UDP

Before we dive into the differences, let's briefly understand what TCP and UDP are:

TCP (Transmission Control Protocol): TCP is like a meticulous courier who ensures that your packages (data) are delivered accurately and in order. It establishes a connection before transmitting data, confirms that the data has reached its destination, and resends it if necessary. TCP is the go-to choice when reliability is paramount.

UDP (User Datagram Protocol): Imagine UDP as a postcard service, it's quick and simple but doesn't guarantee delivery. It sends messages (datagrams) without establishing a connection or confirming that the recipient received them. UDP is ideal when speed is crucial, and a few lost or out-of-order packets are acceptable.

Key Differences Between TCP and UDP

Connection vs. Connectionless: TCP establishes a connection through a three-way handshake before data transfer, ensuring a reliable communication channel. In contrast, UDP is connectionless, sending datagrams without establishing a link, which can lead to faster but less reliable data transmission.

Reliability: TCP provides various mechanisms like sequence numbers, acknowledgments, and retransmissions to ensure reliable data delivery. On the other hand, UDP offers no such assurances, leaving reliability to be handled at the application layer if needed.

Ordering: TCP maintains the order of data packets, reassembling them in the correct sequence at the receiver's end. UDP, however, does not guarantee order, and packets may arrive out of sequence.

Flow Control: TCP implements flow control and congestion control to prevent network congestion, adjusting the data transmission rate based on network conditions. UDP lacks these mechanisms, making it susceptible to congestion and packet loss under high traffic.

Overhead: Due to its connection establishment, error-checking, and flow control features, TCP has a higher overhead compared to UDP, which has minimal headers and no handshaking procedures.

What Developers and DevOps Need to Know

For Developers:

Application Requirements: Choose TCP for applications where data integrity and order are critical, like web servers, emails, or file transfers. Opt for UDP when low latency is essential, and some data loss is tolerable, such as in live audio/video streaming or gaming.

Handling UDP Unreliability: When using UDP, consider implementing your mechanisms for data validation and packet ordering if your application requires it.

Testing and Debugging: Understand that issues like network latency or packet loss will affect TCP and UDP differently. Use network simulation tools to test how your application responds under various conditions.

For DevOps:

Network Monitoring and Troubleshooting: Be adept at using network monitoring tools to track TCP and UDP traffic, identifying bottlenecks or anomalies that could indicate issues.

Configuration and Optimization: Ensure that network devices and firewalls are correctly configured to support the necessary TCP or UDP traffic for your applications. Optimize settings like TCP window size or UDP buffer size based on your network's performance and requirements.

Security: Understand the security implications of both protocols. For example, TCP connections are more susceptible to SYN flood attacks, whereas UDP can be exploited for reflection/amplification attacks. Implement appropriate security measures and monitoring.

Practical Examples

Web Application (TCP):

Imagine developing a web application where data integrity and order are crucial, like an online banking system. TCP is indispensable here, ensuring that every transaction is processed reliably and in the correct sequence.

Live Video Streaming (UDP):

Consider a live sports streaming service where real-time delivery is more critical than perfect data integrity. UDP is the star here, reducing latency and delivering a smooth viewing experience, even if it means occasionally dropping a frame.

Mixed Usage (TCP and UDP):

Think of a multiplayer online game that uses TCP for critical game events like scoring and UDP for real-time player movement. This hybrid approach leverages the strengths of both protocols, balancing reliability and speed.

Conclusion: Embracing the Dual Powers of TCP and UDP

In the intricate dance of network communications, TCP and UDP play pivotal roles, each with its strengths and idiosyncrasies. As developers and DevOps professionals, understanding these protocols allows you to architect robust, efficient, and responsive applications and infrastructures.

Whether you're ensuring the rock-solid reliability of a financial transaction system or pushing the boundaries of real-time streaming, your grasp of TCP and UDP will empower you to make informed decisions, optimizing your applications and networks for the digital age.

So, as you journey back to your coding terminals or monitoring dashboards, remember the unique tales of TCP and UDP. Harness their potential, and watch as your applications and systems thrive in the vibrant ecosystem of networked communications.

Engage, experiment, and excel. The realms of TCP and UDP await your mastery!

---

# AWS VPC IP Versions: IPv4 vs. IPv6

URL: https://ercan.cloud/aws-vpc-ip-versions-ipv4-vs-ipv6/
Date: 2024-02-03T18:59:00+03:00

So, buckle up as we unravel the mysteries of IPv4 and IPv6 within AWS VPC, serving you a platter of insights, examples, and detailed ...


Hello, Cloud Pioneers! Today, we're embarking on an enlightening journey through the realms of AWS VPC, focusing on the intriguing comparison between IPv4 and IPv6. Whether you're orchestrating a sophisticated cloud architecture or keen on understanding the nuanced dance of internet protocols, you've come to the right place. So, buckle up as we unravel the mysteries of IPv4 and IPv6 within AWS VPC, serving you a platter of insights, examples, and detailed explanations.

Introduction to AWS VPC

AWS Virtual Private Cloud (VPC) is your sandbox in the cloud, providing a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network you define. The flexibility and control over your network configuration are phenomenal, enabling you to select IP address ranges, create subnets, and configure route tables and network gateways.

IPv4 in AWS VPC

IPv4 has been the backbone of the internet for decades, providing billions of unique addresses. However, the rapid expansion of the internet and connected devices has nearly exhausted this pool. Within an AWS VPC, IPv4 addresses are represented in CIDR (Classless Inter-Domain Routing) block notation. For instance, a typical IPv4 CIDR block looks like 192.168.0.0/16, representing 65,536 IP addresses.

IPv6 in AWS VPC

Enter IPv6, with its mind-boggling capacity to offer a virtually unlimited number of IP addresses. Designed to future-proof the internet, IPv6 ensures that we won't run out of IP addresses anytime soon. In AWS VPC, an IPv6 address is represented in a more complex notation, such as 2001:db8:1234:1a00::/56, reflecting its hexadecimal nature and larger address space.

Comparing IPv4 and IPv6 in AWS VPC

Now, let's delve into the nitty-gritty of how IPv4 and IPv6 stack up against each other within the context of AWS VPC.

Address Space

IPv4: Offers a 32-bit address space, equating to about 4.29 billion unique addresses. In a world teeming with devices, this number is no longer sufficient.

IPv6: With 128 bits, IPv6 boasts a seemingly infinite pool of addresses, standing strong at 340 undecillion (that's 36 zeros!).

Address Allocation and Assignment

IPv4: You can manually assign IPv4 addresses or let AWS automatically assign them. However, managing a large number of IPv4 addresses can become cumbersome.

IPv6: AWS automatically allocates an IPv6 CIDR block to your VPC and subnets, streamlining address assignment and eliminating the need for complex management.

Communication Protocols

IPv4: Requires NAT (Network Address Translation) for public internet communication, complicating the network architecture and potentially introducing bottlenecks.

IPv6: Designed for end-to-end communication without NAT, IPv6 simplifies routing and enhances security through mandatory IPsec support.

Security

IPv4: Relies heavily on NAT for security, which can create a false sense of security and mask true end-to-end encryption needs.

IPv6: Incorporates IPsec, a suite of protocols for securing internet protocol communications by authenticating and encrypting each IP packet in a data stream.

AWS VPC Features and Compatibility

Both IPv4 and IPv6 in AWS VPC support features like security groups, network ACLs, and route tables. However, not all AWS services support IPv6, and you must design your VPC with dual-stack (IPv4 and IPv6) configurations to ensure compatibility and future-proofing.

Practical Scenarios and Examples

Scenario 1: Hosting a Public Website

Suppose you're hosting a website in your AWS environment. With IPv4, you'd typically assign an Elastic IP (a static IPv4 address) to your instance. In contrast, with IPv6, your instance can have a direct, globally unique IPv6 address, improving accessibility and potentially reducing latency.

Scenario 2: Connecting to the Internet

In an IPv4 VPC, your instances in a private subnet need a NAT gateway for internet access, which can be a bottleneck. With IPv6, your instances can connect directly to the internet without NAT, provided they have public IPv6 addresses, simplifying your network architecture.

Scenario 3: Running a Global Application

When your application needs to serve users worldwide, IPv6 shines with its vast address space, ensuring every user and device can have a unique IP. This global reachability makes IPv6 inherently superior for international applications.

Conclusion: Embracing the Future with IPv6

As our journey concludes, it's clear that while IPv4 has served us well, the future lies with IPv6. Its expansive address space, enhanced security features, and streamlined communication protocols position IPv6 as the linchpin of future networking in AWS VPC and beyond.

Transitioning to IPv6 may seem daunting, but AWS provides the tools and resources to make this shift seamless and rewarding. By understanding and leveraging IPv6 within your AWS VPC, you'll be well-equipped to design robust, scalable, and future-ready cloud architectures.

So, there you have it, a comprehensive dive into the world of AWS VPC's IPv4 and IPv6. As we bid adieu, remember that embracing IPv6 is not just about keeping pace with technology, it's about setting the stage for innovation and ensuring your infrastructure is primed for the opportunities of tomorrow.

Got questions or insights to share about your IPv6 journey? Drop a comment below, and let's keep the conversation going. Happy networking, cloud aficionados!

---

# AWS S3 CORS Settings: A Deep Dive

URL: https://ercan.cloud/aws-s3-cors-settings-a-deep-dive/
Date: 2024-01-05T18:56:43+03:00

Whether you're a seasoned AWS veteran or just starting, mastering CORS settings in S3 can greatly enhance your web applications' functiona...


Hello, Cloud Enthusiasts! Today, we're diving into the world of AWS S3 CORS settings, a topic that, while it might seem daunting at first, is incredibly rewarding to understand. Whether you're a seasoned AWS veteran or just starting, mastering CORS settings in S3 can greatly enhance your web applications' functionality and security.

What is CORS?

Cross-Origin Resource Sharing (CORS) is a security feature that allows you to specify in what ways resources in your bucket can be accessed from a domain different from the one your application is hosted on. In the context of Amazon S3, it controls how files in your bucket are shared with other websites.

Why Do You Need CORS?

Imagine you have a web application hosted at example.com that needs to fetch some assets from your S3 bucket, myawesomebucket.s3.amazonaws.com. Without proper CORS settings, your browser would block these requests due to the same-origin policy, a crucial security mechanism that restricts how a document or script from one origin can interact with resources from another origin.

Setting Up CORS on S3

Now, let's get hands-on and configure CORS for your S3 bucket. AWS has made this process user-friendly, and it can be done via the AWS Management Console, AWS CLI, or SDKs. We'll focus on the Management Console method for its accessibility:

Sign In to the AWS Management Console: Head over to the S3 service page.

Select Your Bucket: Click on the bucket for which you want to set up CORS.

Find the CORS Configuration: Click on the Permissions tab, then find the "Cross-origin resource sharing (CORS)" section.

Edit CORS Configuration: Click on the "Edit" button and you will be presented with a JSON editor where you can define your CORS rules.

Define Your Rules: Here is where you specify which origins can access your resources, which HTTP methods are allowed, and other specifics. For example:

&#91;
    {
        "AllowedHeaders": &#91;"*"],
        "AllowedMethods": &#91;"GET", "POST"],
        "AllowedOrigins": &#91;"https://example.com"],
        "ExposeHeaders": &#91;],
        "MaxAgeSeconds": 3000
    }
]

This rule allows both GET and POST requests from https://example.com, accepts all headers, and doesn't expose any headers. It also sets the cache age to 3000 seconds.

Save Your Configuration: After setting your desired rules, click on "Save changes."

CORS Configuration Elements Explained

AllowedOrigins: Specifies the origins that are allowed to make requests to your bucket. Use ["*"] for all origins.

AllowedMethods: Lists the HTTP methods that can be used (GET, PUT, POST, DELETE, HEAD).

AllowedHeaders: Indicates which headers are allowed in a preflight OPTIONS request.

ExposeHeaders: Specifies which headers are accessible to the script in the client application.

MaxAgeSeconds: Defines how long the results of a preflight request can be cached.

Example Use Cases

Public CDN

If you're using your S3 bucket as a CDN for your public website, you'd set:

&#91;
    {
        "AllowedOrigins": &#91;"*"],
        "AllowedMethods": &#91;"GET"],
        "MaxAgeSeconds": 86400
    }
]

This configuration allows all domains to fetch resources from your bucket using GET requests, with a long cache duration.

Restricted Access

If you want to ensure only your application hosted at example.com can POST data to your bucket:

&#91;
    {
        "AllowedOrigins": &#91;"https://example.com"],
        "AllowedMethods": &#91;"POST"],
        "AllowedHeaders": &#91;"Content-Type"],
        "MaxAgeSeconds": 600
    }
]

This configuration strictly allows POST requests from example.com, accepting only the "Content-Type" header.

Common Pitfalls and Tips

Debugging: Use browser tools or a tool like curl to test and debug CORS settings. Remember, changes might take a few minutes to propagate.

Security: Always restrict your origins and methods as much as possible. Using "*" for origins or allowing all methods can expose your bucket to unnecessary risks.

Testing: Thoroughly test your CORS configuration in a staging environment before pushing to production to avoid any unexpected behavior.

Conclusion

Mastering AWS S3 CORS settings is essential for developing secure and efficient web applications. By understanding and implementing these settings, you can ensure that your S3 resources are accessed exactly as intended, safeguarding your data and optimizing your application's user experience.

Remember, the key to mastering CORS is experimentation and testing. So, roll up your sleeves, dive into your AWS console, and start tweaking those settings. Happy coding!

---

# Importance of Regions and Availability Zones on AWS

URL: https://ercan.cloud/importance-of-regions-and-availability-zones-on-aws/
Date: 2023-12-15T19:12:11+03:00

We're diving into a pivotal aspect of architecting robust, resilient, and efficient applications on Amazon Web Services (AWS): understand...


Today, we're diving into a pivotal aspect of architecting robust, resilient, and efficient applications on Amazon Web Services (AWS): understanding and leveraging AWS Regions and Availability Zones (AZs). This post not only aims to elucidate these key concepts but also to guide you through best practices and practical examples using Terraform, a popular infrastructure-as-code tool. Whether you're a developer, a DevOps engineer, or a cloud architect, mastering these facets of AWS can significantly amplify your applications' performance and reliability.

Unveiling AWS Regions and Availability Zones

At the heart of AWS's global infrastructure are Regions and Availability Zones. These elements are foundational in achieving high availability, low latency, and compliance with regulatory requirements.

AWS Regions

AWS Regions are distinct geographical locations across the world, each hosting multiple isolated Availability Zones. Each Region is a separate entity, ensuring that failures in one Region don't impact another. For businesses, selecting the right Region is crucial for minimizing latency, adhering to data sovereignty laws, and optimizing costs.

AWS Availability Zones

Each AWS Region comprises multiple Availability Zones, which are separate, isolated data centers interconnected with low-latency links. Utilizing multiple AZs in a single Region enables you to build highly available and fault-tolerant applications, as AWS guarantees that AZs are insulated from failures in other AZs.

Key Concepts and Best Practices

When architecting your AWS environment, understanding and strategically deploying across Regions and AZs is paramount. Here's how:

Region Selection: Choose a Region closest to your users to reduce latency. Consider data residency requirements and pricing differences across Regions.

Multi-AZ Deployment: For critical applications, deploy across multiple AZs within a Region to ensure high availability. This strategy protects your applications from data center failures.

Data Replication Across Regions: For global applications or disaster recovery purposes, replicate data across Regions. This ensures business continuity even in the event of a regional disruption.

Region-Aware Services: Some AWS services are region-scoped, while others are global. Understand the scope of each service you use to architect your environment correctly.

Monitoring and Compliance: Use AWS CloudTrail and AWS Config to monitor and ensure compliance with your architectural best practices across Regions and AZs.

Terraform Examples: Implementing Best Practices

Now, let's put theory into practice by illustrating how to implement these concepts using Terraform, an infrastructure-as-code software tool that enables you to define both resource and infrastructure topology in code.

Example 1: Multi-AZ Deployment for High Availability

provider "aws" {
  region = "us-east-1"
}

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "primary" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "us-east-1a"
}

resource "aws_subnet" "secondary" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.2.0/24"
  availability_zone = "us-east-1b"
}

resource "aws_rds_instance" "app_db" {
  engine               = "mysql"
  instance_class       = "db.m4.large"
  allocated_storage    = 100
  db_subnet_group_name = aws_db_subnet_group.app.name
  multi_az             = true
}

resource "aws_db_subnet_group" "app" {
  name       = "main"
  subnet_ids = &#91;aws_subnet.primary.id, aws_subnet.secondary.id]
}

In this example, we deploy an RDS instance across two AZs (us-east-1a and us-east-1b) within the us-east-1 Region for high availability.

Example 2: Data Replication Across Regions

provider "aws" {
  alias  = "primary"
  region = "us-east-1"
}

provider "aws" {
  alias  = "secondary"
  region = "eu-west-1"
}

resource "aws_s3_bucket" "primary_bucket" {
  provider = aws.primary
  bucket   = "my-primary-bucket"
  acl      = "private"
}

resource "aws_s3_bucket" "secondary_bucket" {
  provider = aws.secondary
  bucket   = "my-secondary-bucket"
  acl      = "private"
  versioning {
    enabled = true
  }
  replication_configuration {
    role = aws_iam_role.replication.arn
    rules {
      id     = "replicateAll"
      status = "Enabled"
      destination {
        bucket        = aws_s3_bucket.secondary_bucket.arn
        storage_class = "STANDARD"
      }
    }
  }
}

resource "aws_iam_role" "replication" {
  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": &#91;
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
POLICY
}

In this second example, we create two S3 buckets in different Regions (us-east-1 and eu-west-1) and set up cross-region replication to ensure data availability and durability across geographic locations.

Conclusion: Mastering the AWS Geographical Landscape

Understanding and strategically leveraging AWS Regions and Availability Zones is paramount for designing resilient, efficient, and compliant cloud architectures. By embracing these concepts and applying best practices, you can significantly enhance your applications' performance, availability, and disaster recovery posture.

As we've seen through our Terraform examples, implementing these strategies in code not only boosts efficiency but also ensures consistency and repeatability in your deployments. So, whether you're developing a new application or optimizing an existing one, keep Regions and AZs at the forefront of your architectural decisions.

And with that, we wrap up our deep dive into AWS Regions and Availability Zones. Embrace these insights, implement them in your AWS journey, and watch your cloud infrastructure transform into a paragon of resilience and efficiency. Here's to your success in the cloud!

Got thoughts, questions, or insights on AWS Regions and Availability Zones? Feel free to share in the comments below. Let's cultivate a community of knowledge and innovation! Happy cloud architecting!

---

# The Power of Bash For Loops: Streamlining Your Scripting Tasks

URL: https://ercan.cloud/the-power-of-bash-for-loops-streamlining-your-scripting-tasks/
Date: 2023-12-02T17:30:03+03:00

In the world of scripting and automation, Bash stands out as a versatile and widely-used shell in Unix and Linux systems. Among its ...


In the world of scripting and automation, Bash stands out as a versatile and widely-used shell in Unix and Linux systems. Among its many features, the for loop is a fundamental construct that empowers users to automate repetitive tasks efficiently. In this blog post, we’ll delve into the intricacies of Bash for loops, exploring their syntax, usage, and practical applications. Whether you're a seasoned developer or a beginner, understanding for loops in Bash is a skill that will significantly enhance your scripting capabilities.

Understanding the Basics of For Loops

What is a For Loop?

A for loop is a control flow statement that allows code to be executed repeatedly based on a condition. In Bash, this typically involves iterating over a series of values or a list of items.

Syntax of For Loops in Bash

The basic syntax of a for loop in Bash is as follows:

for variable in list
do
  # commands to be executed
done

Here, the variable takes each value from the list, and the commands inside the loop are executed for each value.

Types of For Loops in Bash

Traditional For Loop

The traditional for loop resembles the syntax found in languages like C or Java:

for (( i=0; i<10; i++ ))
do
  echo $i
done

This loop will print numbers from 0 to 9.

Iterating Over a List

You can also iterate over a list of values:

for color in red green blue
do
  echo "Color: $color"
done

This loop will print each color in the list.

Iterating Over Command Output

Bash allows iterating over the output of a command:

for file in $(ls)
do
  echo "File: $file"
done

This will list all files in the current directory.

Advanced For Loop Concepts

Nested For Loops

For loops can be nested within one another:

for i in 1 2 3
do
  for j in a b c
  do
    echo "$i$j"
  done
done

This prints a combination of numbers and letters.

Using the C-style For Loop

Bash supports a C-style for loop, giving additional control over the iteration:

for (( i=0; i<=10; i+=2 ))
do
  echo "Number: $i"
done

This loop prints even numbers up to 10.

Practical Applications of For Loops

Automating System Administration Tasks

For loops are incredibly useful in system administration for tasks like batch processing of files, user account management, or system monitoring.

Data Processing

Data scientists and analysts can use for loops to automate data processing tasks, such as converting file formats or data extraction.

Web Development

Web developers can use for loops for tasks like automating build processes, testing, or deploying applications.

Best Practices

Keep It Readable

Write clear and readable code. Commenting and proper indentation are crucial for maintaining the code.

Avoid Overuse

While for loops are powerful, overusing them, especially nested loops, can make your script complex and slow.

Test Incrementally

Test your loops with a small set of data before scaling up to avoid surprises.

Conclusion

Bash for loops are a powerful tool in your scripting arsenal. They offer flexibility and efficiency, allowing you to automate a wide range of tasks. By understanding their syntax and applications and adhering to best practices, you can harness their full potential to streamline your scripting tasks.

---

# AWS VPC Design: Integrating Subnets, AZs, and Dual-Stack IP

URL: https://ercan.cloud/aws-vpc-design-integrating-subnets-azs-and-dual-stack-ip/
Date: 2023-11-01T19:31:04+03:00

Virtual Private Cloud (VPC) in AWS, meticulously weaving together subnets, availability zones (AZs), and the dual-stack configuration for...


Welcome, Cloud Architects and Network Savvy Professionals! Today, we embark on an insightful exploration of designing a Virtual Private Cloud (VPC) in AWS, meticulously weaving together subnets, availability zones (AZs), and the dual-stack configuration for IPv4 and IPv6. As organizations strive for robust, scalable, and future-proof network architectures, understanding these components' interplay is paramount. So, let's dive in and unravel the intricacies of AWS VPC design, ensuring you're equipped with the knowledge to architect your network with confidence and foresight.

Foundations of AWS VPC Design

Grasping VPCs and Subnets

At its core, an AWS VPC is a segregated section of the cloud where you can define and control a virtual network. Within this environment, subnets act as subdivisions that segment the VPC's IP address range, enabling you to allocate IP addresses efficiently and organize resources based on functional or security needs.

Embracing Availability Zones

Availability zones (AZs) are distinct locations within a region engineered to be insulated from failures in other AZs, thereby offering enhanced fault tolerance and stability. When designing your VPC, distributing subnets across multiple AZs is crucial for high availability and resilience.

Incorporating Dual-Stack IPv4 and IPv6

The internet is transitioning from IPv4 to IPv6 due to the latter's vast address space and improved network efficiency. AWS supports dual-stack architectures, allowing resources to operate with both IP versions simultaneously, ensuring your infrastructure is future-ready.

Pre-Design Considerations

Before diving into VPC design, it's essential to lay the groundwork:

Understand Your Requirements: Identify your application's architectural needs, considering aspects like traffic flow, data residency, and compliance.

IP Address Planning: Calculate the IP space needed, ensuring sufficient addresses for current and future growth while avoiding overlap with other networks.

Region and AZ Selection: Choose the region closest to your users for optimal latency, and plan to utilize multiple AZs for high availability.

IPv6 Consideration: Acknowledge that IPv6 requires different subnetting and addressing strategies due to its larger address space and hierarchical structure.

Designing Your AWS VPC

Now, let's walk through the steps to create a robust VPC design, integrating subnets, AZs, and dual-stack IP:

Step 1: Create Your VPC

Begin by creating a VPC within your chosen region, specifying the IPv4 CIDR block. AWS also prompts you to associate an IPv6 CIDR block, enabling dual-stack configuration from the get-go.

Step 2: Segment Subnets

Create subnets for each of your VPC's operational segments (e.g., public, private, database layers) across different AZs. Assign IPv4 and IPv6 CIDR blocks to each subnet, respecting the VPC's overall IP allocation. Remember to enable auto-assignment of IPv6 addresses to ensure resources in the subnet can be assigned an IPv6 address.

Step 3: Configure Route Tables

Define route tables to control the traffic flow between subnets, ensuring correct routes for both IPv4 and IPv6 traffic. Typically, you'll need separate routes for internal (within VPC) and external (internet-bound) traffic.

Step 4: Implement Network ACLs and Security Groups

Enhance security by defining network access control lists (NACLs) and security groups. While NACLs offer stateless filtering at the subnet level, security groups provide stateful filtering at the instance level.

Step 5: Deploy NAT Gateways/Instances (IPv4) and Egress-Only Internet Gateways (IPv6)

To enable internet access for private subnets, use NAT devices for IPv4 and egress-only internet gateways for IPv6. Position NAT devices in public subnets and ensure proper route configuration for outbound internet access.

Step 6: Test Connectivity and Compliance

Once your VPC is set up, test both IPv4 and IPv6 connectivity, verifying that resources can communicate as expected and adhere to your design specifications.

Achieving Dual-Stack Networking Excellence

Designing with dual-stack capabilities ensures that your AWS resources are accessible over both IP versions, catering to all clients and future-proofing your network. Here are key points to ensure success:

Monitor and Optimize: Regularly review your VPC configuration, monitoring performance and optimizing routes and IP allocations to adapt to changing needs.

Stay Informed: Keep abreast of AWS features and best practices, integrating new advancements to enhance your VPC design continuously.

Consider Scalability: Always design with scalability in mind, allowing for easy expansion of subnets or integration of additional services.

Conclusion

Mastering the design of AWS VPCs, with integrated subnets, AZs, and dual-stack IP configurations, sets a strong foundation for deploying resilient and future-ready cloud infrastructures. By methodically planning and implementing these elements, you ensure high availability, efficient routing, and the scalability of your AWS resources, positioning your applications for optimal performance and reliability.

Embrace these principles, and watch your network architecture evolve into a model of cloud excellence, ready to meet today's demands and tomorrow's opportunities.

We hope this guide empowers you to design your AWS VPCs with confidence and vision. Should you have insights or questions on crafting impeccable AWS network architectures, feel free to engage in the comments below. Happy designing, cloud visionaries!

---

# 7 Reasons to Consider Getting a Smartwatch

URL: https://ercan.cloud/7-reasons-to-consider-getting-a-smartwatch/
Date: 2023-10-31T19:16:43+03:00


You have probably noticed a fair few people walking around with smartwatches. The trend might be causing you to scratch your head, trying to figure out what the fuss is all about.

There are multiple brands that are decently known for manufacturing smartwatches, though Apple (unsurprisingly) is probably at the top.

Whether it is curiosity or peer pressure, you might be thinking about getting a smartwatch for yourself as well. And if that is the case, there are quite a few reasons to justify your decision. Smartwatches offer more than meets the eye, and we will cover exactly that in this article.

A Fashion Statement

Let’s get this one right out of the way. Smartwatches are not as noticeable as Rolexes or other hand-watches, but the fashion accessory aspect is still there.

If anything, a smartwatch represents one’s modern outlook on things and how important of a role technology plays in their life. 

Besides, it also helps when you have multiple brands and models to choose from. If you are not too keen on a particular watch, you can delve deeper into researching the market until you find something that you like.

Entertainment

Smartwatches do not have as much storage as other devices like computers. Nor is it as simple to transfer files from another appliance. For example, you can Airdrop from iPhone to Mac with relative ease, but depending on what smartwatch you have, there might be some issues.

Nevertheless, the drawback does not mean that you cannot make the most out of the gadget entertainment-wise.

Technology is advancing fast, and it is not surprising to find a plethora of various applications available on a smartwatch. Listening to music or podcasts, watching videos and other media, or accessing that little bit of entertainment stored directly on the smartwatch are viable options.

It is also worth noting that gaming is another idea that is gaining more and more momentum. Some manufacturers are trying to develop smartwatches dedicated to gaming specifically. 

Safety

Smartwatches come with a built-in GPS tracker, which functions as a safety net not just for you but others. You can have your children wear one, and if something happens, you can detect their location with the help of GPS.

Naturally, you want to avoid and not think about such issues, but you cannot be certain about when something might happen, and having an accessory to back up upon when in need is often underrated.

Notifications

Are you away from your smartphone or laptop and want to make sure that you do not miss an important message or a phone call? Syncing your smartwatch with other devices enables you to receive notifications directly on the smartwatch, which makes it much easier to avoid potential problems related to work or personal matters.

Battery Usage

Speaking of syncing the smartwatch with other devices, you can also expect to encounter fewer battery issues on your smartphone.

If the mobile device is not overburdened as much due to the smartwatch taking some of the processes to itself, there is no need to recharge the smartphone as often. 

Of course, this specific benefit might not seem like that big of a deal, but it is still another point in favor of smartwatches.

Health Monitor

You can keep a constant track of your health. Health awareness is often the go-to reason for people to invest in a smartwatch.

Stress levels, heart rates, sleep patterns, and other critical data become accessible when you “connect” your body with a smartwatch. As soon as something goes wrong, you will receive an alert and can take the necessary steps to reduce and eliminate a health hazard.

Traveling

Traveling becomes more convenient when you have a smartwatch on your wrist. The gadget is small, so you do not have to worry about it taking up storage space in your luggage. But the best thing is having access to so many neat things that are necessary when traveling.

GPS to tell you where to go, a map system, reviews about establishments you want to visit, and even translation services to communicate with strangers who do not understand your language. 

Closing Thoughts

To sum it all up, it should be pretty clear why smartwatches are so popular. The benefits mentioned in this article should change the mind of those who are still on the fence about buying a smartwatch themselves.

And considering how fast new and improved smartwatch versions are being released every year, the future seems even brighter, and there might be even more reasons to get this particular device later down the road.

---

# How to Enable SSH on Ubuntu: A Step-by-Step Guide

URL: https://ercan.cloud/how-to-enable-ssh-on-ubuntu-a-step-by-step-guide/
Date: 2023-10-19T18:15:00+03:00

Secure Shell (SSH) is an essential tool for system administrators, developers, and IT professionals. It provides a secure way to access ...


Introduction

Secure Shell (SSH) is an essential tool for system administrators, developers, and IT professionals. It provides a secure way to access a remote server, making it an invaluable resource for managing systems remotely. For users of Ubuntu, one of the most popular Linux distributions, setting up SSH can greatly enhance the system's usability and security. This blog post will guide you through the process of enabling SSH on Ubuntu, ensuring a seamless and secure remote access experience.

What is SSH?

SSH, or Secure Shell, is a network protocol that allows for secure data communication, remote shell services, and command execution. It is widely used by system administrators for managing systems and applications remotely. SSH encrypts the data transmitted, ensuring that sensitive information remains secure from unauthorized access.

Why Enable SSH on Ubuntu?

Enabling SSH on Ubuntu allows for secure remote access to your Ubuntu machine. This is particularly useful for managing servers, automating tasks, and transferring files securely over unsecured networks.

Step-by-Step Guide to Enabling SSH on Ubuntu

Update Your Ubuntu System Before installing any new software, it's a good practice to update your system. Open your terminal and run the following commands:

sudo apt update
sudo apt upgrade

This will ensure that all your system's packages are up to date.

Install the OpenSSH Server Ubuntu uses OpenSSH, an open-source SSH tool. To install it, run:

sudo apt install openssh-server

After the installation is complete, the SSH service will start automatically.

Verify SSH Service Status To check if the SSH server is running, use:

sudo systemctl status ssh

You should see a status indicating that the service is active and running.

Configure SSH (Optional) The default configuration of SSH is generally sufficient for most users. However, you can make changes to the configuration file located at /etc/ssh/sshd_config. Some common configurations include changing the default SSH port, disabling root login, and limiting user access. Remember to restart the SSH service after making any changes:

sudo systemctl restart ssh

Adjust Firewall Settings If you have a firewall enabled, you need to allow SSH traffic. For Ubuntu's default UFW firewall, run:

sudo ufw allow ssh

This will allow traffic on port 22, the default SSH port.

Connect to Your Ubuntu System via SSH To connect to your Ubuntu machine from another computer, use the SSH command:

ssh username@your_server_ip

Replace username with your user account on the Ubuntu system and your_server_ip with the machine's IP address.

Securing Your SSH Connection

Use Strong Passwords: Ensure that all user accounts on the system have strong, unique passwords.

Disable Root Login: It's a security best practice to disable direct root login. Configure sudo for administrative tasks.

Change the Default SSH Port: Changing the default port (22) to a non-standard port can reduce the risk of automated attacks.

Use SSH Key Authentication: For enhanced security, set up SSH key-based authentication instead of password-based authentication.

Conclusion

Enabling SSH on Ubuntu is a straightforward process that opens up a world of possibilities for secure remote management of your system. By following the steps outlined in this guide, you can ensure that your SSH setup on Ubuntu is secure and efficient. Remember to follow best practices for securing your SSH connection to safeguard against unauthorized access.

Can I use SSH with a graphical user interface (GUI)? Yes, SSH can be used to forward X11 sessions, allowing you to use GUI applications remotely.

  How do I transfer files over SSH? You can use tools like scp or rsync to securely transfer files over SSH.

  What should I do if I'm unable to connect via SSH? Check your firewall settings, ensure the SSH service is running, and verify your network connection.

  Is it safe to open SSH to the internet? While SSH is secure, exposing any service to the internet increases risk. Use strong passwords, SSH keys, and consider setting up a VPN for added security.

  
By understanding and implementing SSH on Ubuntu, you'll enhance your system's capabilities while maintaining a high level of security. Happy SSHing!

---

# Vim: Mastering the Essentials for Efficient Editing

URL: https://ercan.cloud/vim-mastering-the-essentials-for-efficient-editing/
Date: 2023-09-25T17:53:44+03:00

In the world of text editing, Vim stands out as a powerful, keyboard-centric editor that prioritizes efficiency and speed. Originating ...


Introduction

In the world of text editing, Vim stands out as a powerful, keyboard-centric editor that prioritizes efficiency and speed. Originating from the Unix environment, Vim (Vi Improved) is known for its versatility and the high level of productivity it offers once you get past its steep learning curve. In this article, we'll delve into the core Vim commands like copy, undo, redo, jumping between words, and more, which are essential for anyone looking to enhance their text editing skills. Designed for SEO optimization and readability, this guide aims to simplify Vim for beginners and intermediate users alike.

What is Vim?

Vim is a highly configurable text editor built to enable efficient text editing. It's an improved version of the vi editor distributed with most UNIX systems. Vim is often used for writing and editing code, but it's also great for writing articles or even configuring systems.

Why Use Vim?

Speed: Vim is designed for using the keyboard efficiently, minimizing the need to use the mouse.

Availability: It's available on almost all Unix-like systems and has been ported to other operating systems.

Customization: Vim is highly customizable, allowing users to create their own commands and automation.

Powerful: With a wide range of commands, Vim can handle complex editing tasks with ease.

Basic Vim Commands

Opening and Closing Files

Open Vim: Just type vim in the terminal.

Open a File: Type vim filename.

Save a File: Press Esc and then :w.

Exit Vim: Press Esc and then type :q. To save and exit, type :wq.

Navigation

Move Cursor: Use the h, j, k, l keys to move left, down, up, and right, respectively.

Jump Words: Use w to jump forwards to the start of a word, e to jump to the end of a word, and b to jump backwards to the start of a word.

Editing

Insert Mode: Press i to enter insert mode. Press Esc to return to command mode.

Copy (Yank): Press yy to copy a line or y followed by a movement command to copy that range.

Paste: Press p to paste after the cursor, or P to paste before.

Undo and Redo: Press u to undo and Ctrl-r to redo.

Advanced Commands

Search: Press / followed by your search term and Enter. Use n and N to navigate through the search results.

Replace: Use :%s/old/new/g to replace all occurrences of 'old' with 'new'.

Tips for Beginners

Start with the Basics: Focus on a few commands at a time.

Practice: The more you use Vim, the more comfortable you'll become.

Customize Your Environment: As you get more comfortable, start adding custom configurations in your .vimrc file.

Common Challenges and Solutions

Remembering Commands: Vim commands are not always intuitive. Consider using a Vim cheat sheet.

Switching Modes: Remembering to switch between insert and command mode can be tricky at first. Practice is key.

Advanced Vim Usage

As you become more comfortable with Vim, you can explore advanced features like split screens, tabs, macros, and custom configurations. Vim also supports a wide range of plugins for different languages and tools, enhancing its functionality.

Conclusion

Vim might have a reputation for being difficult for beginners, but it's a powerful tool once you get the hang of it. Understanding the basic commands for copying, undoing, redoing, and jumping between words is a great start. Remember, the key to mastering Vim is patience and practice. With time, you'll find that Vim can significantly speed up your editing process, making it a worthwhile investment of your time to learn.

FAQs

Is Vim suitable for beginners? Yes, while it has a steep learning curve, starting with basic commands and gradually learning more can make Vim accessible to beginners.

  How do I customize Vim? Vim can be customized by editing the .vimrc file in your home directory, where you can add your own settings, mappings, and plugins.

  Are there any good resources to learn Vim? There are many online resources, including tutorials, forums, and videos. The built-in vimtutor command is a great starting point.

  Can Vim be used for programming? Absolutely. Vim is a popular choice among programmers due to its efficiency and the extensive range of plugins available for different programming languages

  
By mastering these fundamental Vim commands and techniques, you're setting yourself up for a more productive and efficient editing experience. Whether you're coding, writing content, or configuring systems, Vim's powerful features can cater to a wide range of text editing needs. Remember, the journey to becoming proficient in Vim is gradual. Embrace the learning curve, and soon, you'll find yourself seamlessly navigating and editing files with a speed and efficiency that few text editors can match.

Building Vim Skills

Developing Muscle Memory

Regular Practice: Incorporate Vim into your daily workflow to build muscle memory.

Incremental Learning: Focus on learning new commands one at a time.

Exploring Plugins

Language-specific Plugins: Enhance your programming with language-specific tools.

Productivity Plugins: Explore plugins that add functionality like file explorers, git integration, and syntax highlighting.

Customizing Vim

Tailor Vim to your needs by customizing keybindings and settings.

Experiment with different color schemes and layouts to create a comfortable working environment.

Community and Resources

Vim boasts a vibrant community and a wealth of resources. Engaging with community forums, following Vim blogs, and participating in online discussions can provide valuable insights and tips. Additionally, numerous online tutorials and courses are available for those looking to deepen their understanding of Vim.

Conclusion

In conclusion, Vim is more than just a text editor; it's a tool that, once mastered, can significantly enhance your productivity and efficiency. By starting with the basics and gradually exploring more complex features and customizations, you can transform Vim into an indispensable part of your toolkit. Embrace the challenge, and enjoy the journey of becoming a Vim power user!

By mastering these fundamental Vim commands and techniques, you're setting yourself up for a more productive and efficient editing experience. Whether you're coding, writing content, or configuring systems, Vim's powerful features can cater to a wide range of text editing needs. Remember, the journey to becoming proficient in Vim is gradual. Embrace the learning curve, and soon, you'll find yourself seamlessly navigating and editing files with a speed and efficiency that few text editors can match.

Building Vim Skills

Developing Muscle Memory

Regular Practice: Incorporate Vim into your daily workflow to build muscle memory.

Incremental Learning: Focus on learning new commands one at a time.

Exploring Plugins

Language-specific Plugins: Enhance your programming with language-specific tools.

Productivity Plugins: Explore plugins that add functionality like file explorers, git integration, and syntax highlighting.

Customizing Vim

Tailor Vim to your needs by customizing keybindings and settings.

Experiment with different color schemes and layouts to create a comfortable working environment.

Community and Resources

Vim boasts a vibrant community and a wealth of resources. Engaging with community forums, following Vim blogs, and participating in online discussions can provide valuable insights and tips. Additionally, numerous online tutorials and courses are available for those looking to deepen their understanding of Vim.

Conclusion

In conclusion, Vim is more than just a text editor; it's a tool that, once mastered, can significantly enhance your productivity and efficiency. By starting with the basics and gradually exploring more complex features and customizations, you can transform Vim into an indispensable part of your toolkit. Embrace the challenge, and enjoy the journey of becoming a Vim power user!

FAQs Continued

How long does it take to become proficient in Vim? Proficiency in Vim depends on how frequently you use it and your dedication to learning. Regular daily use can lead to proficiency in a few months.

  Can Vim be used on Windows? Yes, Vim is available on Windows and can be installed either as a standalone application or as part of environments like Git Bash or WSL (Windows Subsystem for Linux).

  Is Vim only for text editing, or can it be used for other purposes? While Vim is primarily a text editor, its versatility allows it to be used for a wide range of purposes, including programming, system administration, and even as a simple IDE with the right plugins and configurations.

  
By understanding and utilizing the power of Vim, you open up a world of efficient and effective text editing, setting a foundation for a more productive workflow.

---

# Understanding CAA DNS Records: What, Why, and How?

URL: https://ercan.cloud/understanding-caa-dns-records-what-why-and-how/
Date: 2023-09-25T13:27:12+03:00

CAA, or Certificate Authority Authorization, is a type of DNS record that allows domain owners to specify which Certificate ...


The digital world thrives on security and trust. One of the foundational aspects of this trust is the SSL/TLS certificate, the backbone of HTTPS. As the internet matures, so does the need for improving the certificate issuance process. Enter the CAA (Certificate Authority Authorization) DNS record.

What is a CAA DNS Record?

CAA, or Certificate Authority Authorization, is a type of DNS record that allows domain owners to specify which Certificate Authorities (CAs) are allowed to issue certificates for their domain. In essence, it gives domain owners a say in who can and cannot issue certificates for their domains.

For instance, a simple CAA record might look like this:

example.com. CAA 0 issue "amazon.com"
example.com. CAA 0 issue "comodo.com"
example.com. CAA 0 issue "letsencrypt.org"

This means only Let's Encrypt is authorized to issue certificates for example.com.

Why Do We Need CAA DNS Records?

Enhanced Security: CAA records help protect domains from mistakenly being issued rogue or misissued certificates. By specifying a list of authorized CAs, domain owners can reduce the risk of man-in-the-middle attacks.

Control: CAA records give domain owners greater control over their SSL/TLS certificates. If a domain owner only trusts certain CAs, they can ensure only those CAs issue certificates for their domain.

Audit Trail: If a CA receives a request to issue a certificate for a domain that has a CAA record not listing that CA, the CA will document the request and notify the domain owner, providing a trail of any unauthorized certificate requests.

Pros of Using CAA DNS Records:

Prevention of Unauthorized Certificates: By specifying which CAs can issue certificates, you can prevent unauthorized entities from mistakenly or maliciously obtaining a valid certificate for your domain.

Flexibility: CAA records can be set for each subdomain, offering granularity. For example, you can have different CAs for blog.example.com and shop.example.com.

Compatibility: CAA checking is now mandatory for all CAs. This means any CA must respect the CAA records before issuing a certificate.

Cons of Using CAA DNS Records:

Maintenance: If you decide to change your CA or use multiple CAs, you'll need to update your CAA records accordingly.

Propagation Delays: Like all DNS changes, updates to CAA records can take time to propagate, which might delay the issuance of new certificates.

Potential for Misconfiguration: Incorrect CAA records can prevent legitimate certificate issuance.

What Were We Doing Before?

Before the adoption of CAA records, there was no standardized method for domain owners to express their preference for certificate issuers. This means any CA could issue a certificate for any domain, as long as they could validate ownership. This left room for errors, misissuances, and potential security vulnerabilities.

A Practical Example: AWS ACM and CloudFlare

Let's say you're trying to create an SSL certificate via AWS ACM for ercanermis.com and *.ercanermis.com using the email validation method.

If ercanermis.com already has a CAA record set (for example, for another CA) but doesn't have a wildcard setup on CloudFlare, you might run into issues. I'm saying CloudFlare because, I'm using for my DNS records management and other features. You can see the example below. 

To resolve this, you would need to:

Add a CAA record for AWS ACM (amazon.com) for your domain on your DNS Manager like if it's not on Route53.

Ensure that you also cover the wildcard subdomains if you're trying to secure them.

You can see the CloudFlare records and AWS ACM successfully issued a bot domain for ercanermis.com and *.ercanermis.com.

Remember, if you have existing CAA records, adding a new one doesn't remove the old ones unless you explicitly do so.

Conclusion

CAA records are a valuable tool in the domain owner's arsenal for maintaining the integrity and security of their websites. While they add an extra layer of management, the security benefits, control, and peace of mind they offer often outweigh the cons.

As the digital landscape evolves, tools like CAA will become even more essential in ensuring a safe and trustworthy internet. If you're a domain owner, it might be time to look into setting up CAA records for your domain.

---

# Mastering AWS VPC: An Introduction to Virtual Private Cloud Routing

URL: https://ercan.cloud/mastering-aws-vpc-an-introduction-to-virtual-private-cloud-routing/
Date: 2023-08-13T12:03:24+03:00

The world of cloud computing has never been more accessible or diverse. As organizations migrate to the cloud, AWS ...


The world of cloud computing has never been more accessible or diverse. As organizations migrate to the cloud, AWS (Amazon Web Services) stands out as a go-to solution, especially its VPC (Virtual Private Cloud) service. In this introduction, we delve deep into AWS VPC routing, demystifying its core concepts and how they contribute to an optimized, secure cloud environment.

What is AWS VPC?

At its core, AWS VPC is a customizable, isolated section of the Amazon Web Services cloud where users can launch AWS resources within a virtual network. Think of it as your private chunk of the AWS cloud, where you control IP address ranges, subnets, route tables, and network gateways.

Why is VPC Routing Important?

VPC routing plays a pivotal role in determining how traffic flows between different subnets, both inside your VPC and out to other networks or the internet. Mastery over routing ensures secure, efficient, and reliable communication between resources.

Core Concepts of AWS VPC Routing:

Route Tables: These act as routers, determining where network traffic is directed. Each subnet in your VPC must be associated with a route table.

Subnets: Partitions of your VPC's IP address range where you can place groups of isolated resources. Internet Gateway: This allows your VPC to communicate with the internet. It’s vital if you want resources in your VPC (like EC2 instances) to access the internet directly.

NAT Devices: Network Address Translation devices (either NAT Gateway or NAT Instances) allow instances in a private subnet to initiate outbound traffic to the internet.

VPC Peering: This is a connection between two VPCs, enabling traffic to flow between them as if they were in the same network.

Getting Started with VPC Routing:

Design your VPC: Before diving in, plan out your VPC's CIDR blocks, considering the number of IP addresses you'll need and future scalability.

Set up subnets: Design public and private subnets based on your architectural needs. A public subnet has direct access to the internet via an Internet Gateway, while a private subnet does not. 

Configure route tables: Create route rules that define how traffic flows. For example, a route in your table can direct all traffic (0.0.0.0/0) to an Internet Gateway.

Integrate NAT devices: For private subnets, NAT devices help instances communicate with the internet for tasks like updates. Monitor and optimize: AWS offers tools like VPC Flow Logs to help you monitor traffic, which is crucial for optimizing and securing your network.

In Conclusion

Mastering AWS VPC routing is fundamental for anyone diving into the AWS ecosystem. The power of VPC lies in its flexibility, allowing you to design a virtual networking environment tailored to your specific needs. As you become more acquainted with VPC routing principles, you'll find yourself better equipped to design robust, scalable, and secure cloud architectures.

If you found this introduction to AWS VPC routing valuable, please share it and stay tuned for more deep dives into the world of cloud computing!

---

# Understanding the New AWS Public IPv4 Address Charge and Public IP Insights

URL: https://ercan.cloud/understanding-the-new-aws-public-ipv4-address-charge-and-public-ip-insights/
Date: 2023-07-30T18:57:40+03:00


Hello everyone,

I wanted to share some important updates from Amazon Web Services (AWS) that could impact how you use their services. AWS has announced a new charge for public IPv4 addresses, effective from February 1, 2024. This change will see a charge of $0.005 per IP per hour for all public IPv4 addresses, whether they are attached to a service or not.

Why the New Charge?

IPv4 addresses are becoming an increasingly scarce resource. Over the past five years, the cost to acquire a single public IPv4 address has risen by more than 300%. This new charge reflects AWS's own costs and is intended to encourage users to be more frugal with their use of public IPv4 addresses. It's also a nudge towards accelerating the adoption of IPv6 as a modernization and conservation measure.

Who Does This Affect?

This change applies to all AWS services, including Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (RDS) database instances, Amazon Elastic Kubernetes Service (EKS) nodes, and other AWS services that can have a public IPv4 address allocated and attached. This is applicable across all AWS regions.

What About the AWS Free Tier?

The AWS Free Tier for EC2 will include 750 hours of public IPv4 address usage per month for the first 12 months, effective from February 1, 2024. You will not be charged for IP addresses that you own and bring to AWS using Amazon BYOIP.

Monitoring Your Usage

Starting from now, your AWS Cost and Usage Reports will automatically include public IPv4 address usage. When this price change goes into effect next year, you will also be able to use AWS Cost Explorer to see and better understand your usage.

Accelerating Your Adoption of IPv6

AWS is encouraging users to consider accelerating their adoption of IPv6. They have a new blog post showing how to use Elastic Load Balancers and NAT Gateways for ingress and egress traffic, while avoiding the use of a public IPv4 address for each instance that you launch.

Introducing Public IP Insights

To make it easier for you to monitor, analyze, and audit your use of public IPv4 addresses, AWS is launching Public IP Insights, a new feature of Amazon VPC IP Address Manager that is available at no cost. This tool will give you a better understanding of your security profile and help you make efficient use of public IPv4 addresses.

Final Thoughts

By using the new IP Insights tool and following the guidance shared by AWS, you should be ready to update your application to minimize the effect of the new charge. You may also want to consider using AWS Direct Connect to set up a dedicated network connection to AWS.

For more information on how to make the best use of public IPv4 addresses, be sure to read AWS's new blog post, "Identify and Optimize Public IPv4 Address Usage on AWS".

Stay tuned for more updates and insights!

---

# Docker Multi-Stage Builds: An In-depth Guide

URL: https://ercan.cloud/docker-multi-stage-builds-an-in-depth-guide/
Date: 2023-07-29T09:09:39+03:00


Introduction

Docker has revolutionized the way we develop, package, and deploy applications. It provides a consistent environment for applications to run, from development to production, reducing the "it works on my machine" problem. One of the most powerful features Docker has introduced is multi-stage builds. This feature helps us create lean, efficient containers without the usual hassle. In this blog post, we will delve into Docker multi-stage builds, their benefits, and best practices. We will also provide an example Dockerfile to illustrate the concept.

What is Docker Multi-Stage Build?

Docker multi-stage build is a feature that allows you to use multiple FROM statements in your Dockerfile. Each FROM statement can use a different base image and starts a new stage of the build. You can selectively copy artifacts from one stage to another, leaving behind everything you don't need in the final image. This allows you to create smaller, more efficient images by separating the building and packaging processes into different stages.

Why Use Docker Multi-Stage Builds?

Before the introduction of multi-stage builds, creating lean Docker images was a bit of a hassle. You had to create separate Dockerfiles for building and running applications, or install unnecessary tools in your production images. Multi-stage builds solve these problems by allowing you to use one Dockerfile to create efficient images. Here are some reasons why you should use Docker multi-stage builds:

Smaller Image Size: By copying only the necessary files from the build stage to the final image, you can significantly reduce the size of your Docker images. Smaller images are faster to push and pull from registries, and they use less disk space.

Separation of Concerns: With multi-stage builds, you can separate the build-time dependencies from the runtime dependencies. This makes your Dockerfile easier to read and maintain.

Security: Smaller images have a smaller attack surface. By excluding unnecessary tools and files, you reduce the potential for security vulnerabilities.

Best Practices

Use Specific Base Images: For each stage, use the most specific base image that includes only what you need for that stage. For example, use a Node.js image for a build stage that involves a Node.js application, and an Alpine image for a lightweight final stage.

Optimize Layer Creation: Docker builds images in layers. To make your builds faster and your images smaller, try to minimize the number of layers by combining commands using &&.

Clean Up After Yourself: In the build stage, clean up unnecessary files and artifacts after you're done with them. This will make the build cache smaller and faster.

Example Dockerfile

Here's an example Dockerfile that demonstrates a multi-stage build for a Node.js application:

Dockerfile

# ---- Base Node ----
FROM node:14 AS base
WORKDIR /usr/src/app
COPY package*.json ./

# ---- Dependencies ----
FROM base AS dependencies
RUN npm install

# ---- Test ----
# run linters, setup and tests
FROM dependencies AS test
COPY . .
RUN npm run test

# ---- Build ----
FROM dependencies AS build
COPY . .
RUN npm run build

# ---- Release ----
FROM node:14-alpine AS release
WORKDIR /usr/src/app
COPY --from=build /usr/src/app/dist ./dist
EXPOSE 8080
CMD &#91;"node", "dist/index.js"]

In this Dockerfile, we have five stages: base, dependencies, test, build, and release. The base stage copies the package.json files. The dependencies stage installs the Node modules. The test stage runs the tests. The build stage builds the application. Finally, the release stage creates the final image that will be used in production. Only the necessary files are copied to the final image, resulting in a smaller, more efficient image.

Conclusion

Docker multi-stage builds are a powerful tool for creating efficient Docker images. They allow you to separate your build and runtime environments, reduce your image size, and improve security. By following best practices and using a well-structured Dockerfile, you can take full advantage of this feature. Happy Dockering!

---

# Mastering Docker Run Command: A Comprehensive Guide

URL: https://ercan.cloud/mastering-docker-run-command-a-comprehensive-guide/
Date: 2023-07-29T09:02:20+03:00


The use of Docker in contemporary software development is widespread, and for good reason. Docker helps developers build lightweight and portable software containers that simplify application deployment. One of the most powerful commands in Docker's command-line interface (CLI) is the docker run command. However, its vast number of options can make it a bit daunting for beginners.

In this blog post, we will demystify the docker run command and explain how to leverage its capabilities to run Docker containers effectively. Let's get started!

The Docker Run Command

At its most basic, the docker run command creates a new container from a Docker image and starts it. However, this command offers a multitude of options to customize your containers according to your application's needs. Here's how the command structure looks:

docker run [OPTIONS] IMAGE[:TAG|@DIGEST] [COMMAND] [ARG...]

Now, let's delve deeper into some of the most commonly used options:

Name (--name)

Assigning a custom name to your Docker container can be incredibly useful, especially when you're dealing with multiple containers. You can specify a name for your Docker container using the --name flag:

docker run -d --name my_container ubuntu

Port Mapping (-p or --publish)

Docker allows you to map a network port inside your Docker container to a port on your host machine using the -p or --publish flag. This capability is essential when deploying services that need to be accessible over a network:

docker run -d -p 8080:80 ubuntu

Volume Mounting (-v or --volume)

To persist data generated and used by your Docker containers or to provide data from your host to the container, Docker provides volumes. You can mount a volume using the -v or --volume flag:

docker run -d -v /path/on/host:/path/in/container ubuntu

Environment Variables (-e or --env)

You can set environment variables that the process inside your Docker container can use. These variables are a good way to provide configuration settings to your application:

docker run -d -e "ENV_VAR=value" ubuntu

Link (--link)

The --link option allows containers to discover each other and securely transfer information about one container to another container.

docker run -d --link database:db ubuntu

Memory (-m or --memory)

Limiting a container's memory usage can be critical when managing resources on your Docker host. The -m or --memory flag allows you to limit a container's memory usage:

docker run -d -m 512m ubuntu

Network (--net)

The --net option allows you to specify the networking mode for the container. For instance, using --net=host makes the container use the host's network stack:

docker run -d --net=host ubuntu

Restart (--restart)

The --restart flag allows you to specify a restart policy for how Docker should handle container exits. The always option ensures the container is always restarted, maintaining high availability:

docker run -d --restart=always ubuntu

User (--user or -u)

The --user option allows you to specify the user that runs the command in the container. This can be useful to ensure correct file permissions or to enhance the container's security:

docker run -d --user=username ubuntu

Working Directory (-w or --workdir)

The -w or --workdir option sets the working directory inside the container for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile:

docker run -d -w /path/to/dir/ ubuntu

Now, let's see an example of a docker run command that includes multiple options:

docker run -d \
--name my_container \
-p 8080:80 \
-v /path/on/host:/path/in/container \
-e "ENV_VAR=value" \
--link database:db \
-m 512m \
--net=host \
--restart=always \
--user=username \
-w /path/to/dir/ \
ubuntu

This command chain gives you a powerful, flexible way to configure and manage Docker containers to suit various needs. Remember, not all options need to be used together in all cases; adjust the command to fit your specific requirements.

Wrapping Up

In conclusion, the docker run command is a versatile tool that empowers developers to efficiently manage and control Docker containers. By understanding and effectively using these options, you can significantly enhance your Docker workflow and productivity.

This guide just scratches the surface of Docker's capabilities. I encourage you to explore the Docker documentation and experiment with these options to get a feel for what's possible with Docker.

Whether you're a developer, DevOps professional, or IT administrator, mastering the docker run command is an essential skill that will surely serve you well in your Docker journey.

Happy Dockering!

---

# MySQL 8: Understanding and Fixing Error 1819

URL: https://ercan.cloud/mysql-8-understanding-and-fixing-error-1819/
Date: 2023-07-14T15:25:56+03:00

As a beginner in MySQL, you might encounter an error message that reads "ERROR 1819 (HY000): Your password does not satisfy the current ...


As a beginner in MySQL, you might encounter an error message that reads "ERROR 1819 (HY000): Your password does not satisfy the current policy requirements." This error is not a bug, but a built-in security feature of MySQL that ensures users set strong passwords. This tutorial will guide you through understanding this error and how to fix it.

What Triggers the Error?

When setting up a password for the MySQL root user, you may be prompted to enable the VALIDATE PASSWORD component. If enabled, this component checks the strength of the password you provide. If your password is considered weak, you'll encounter the error 1819.

For example, if you try to create a user with a weak password like this:

mysql> create user 'ostechnix'@'localhost' identified by 'mypassword';

You'll see the error:

ERROR 1819 (HY000): Your password does not satisfy the current policy requirements

This error will persist until you provide a password that meets the current password policy requirements or disable the VALIDATE PASSWORD component.

Understanding Password Validation Policy

MySQL enforces three levels of password validation policy when the VALIDATE PASSWORD plugin is enabled:

LOW: Password must be at least 8 characters long.

MEDIUM: Password must be at least 8 characters long, include numeric, mixed case, and special characters.

STRONG: Password must be at least 8 characters long, include numeric, mixed case, special characters, and dictionary file.

To find the current password policy level, run the following command:

mysql> SHOW VARIABLES LIKE 'validate_password%';

The output will show the currently enforced password level. For instance, if the password level is Medium, your password should be 8 characters long and include a number, mixed case, and special characters.

How to Fix the Error

There are two main ways to fix the error 1819:

1. Change the Password to Meet the Policy

The first method is to set a password that meets the current password validation policy. For instance, if the policy is set to Medium, you could set a password like 'Password123#@!' which includes a number, lowercase, uppercase, and special characters.

2. Change the Password Validation Policy

The second method is to change the password validation policy to a lower level. To do this, run the following command from the MySQL prompt:

mysql> SET GLOBAL validate_password.policy=LOW;

Then, check if the password validation policy has been changed to low:

mysql> SHOW VARIABLES LIKE 'validate_password%';

Now, you can create a user with a weak password. To revert back to the MEDIUM level policy, simply run this command:

mysql> SET GLOBAL validate_password.policy=MEDIUM;

If the password policy doesn't change, exit from the MySQL prompt and restart the MySQL service.

Disabling Password Validation Policy

If you wish to create users with weak passwords, you can disable the Validate Password component altogether and re-enable it back after creating the users. However, it's not recommended to disable password policy or use weak passwords. Always use a strong password with more than 8 characters including a number, mixed case, and special characters.

Conclusion

In this tutorial, we've covered the common MySQL error 1819 and how to fix it. We've also discussed how to disable the password policy to allow weak passwords in some cases. However, it's always recommended to use a strong password for better security. Happy learning!

---

# Ctrl+C and Ctrl+V party with GitHub Clone Helper!

URL: https://ercan.cloud/ctrlc-and-ctrlv-party-with-github-clone-helper/
Date: 2023-06-24T23:53:09+03:00

When you copy the clone URL, you're actually copying the entire 'git clone' command. It's the Ctrl+C and Ctrl+V party you always wanted ...


Hello, fellow keyboard warriors and terminal whisperers! 🤓

Ever find yourself lost in the infinite sea of repositories on GitHub, and you’ve to clone a bunch of them? Now, we all know how it goes: copy the clone URL, open your terminal, type ‘git clone’, paste the URL, and hit Enter. Sounds easy, right? But what about the 50th time? The 100th? The 1000th?! Man, even the thought of it makes my fingers cramp! 😵‍💫

Now, we all love a good typing session (the rhythmic click-clack is oddly satisfying), but typing ‘git clone’ before every URL, every single time? That’s where we draw the line. It’s like having to say “Simon says” before every command. Simon, we get it, you’re in charge! 🙄

Enter the GitHub Clone Helper! This nifty little extension is the superhero we didn’t know we needed. It swoops in and puts an end to our ‘git clone’ woes. It’s like that awesome sidekick who finishes your sentences, only this time, it’s finishing your commands. Now, who wouldn’t want that? 🦸‍♀️

GitHub Clone Helper is no ordinary extension. It’s a hardworking, diligent little elf that automatically adds ‘git clone’ before the clone URLs on GitHub. And it doesn’t just show it; it means it. When you copy the clone URL, you’re actually copying the entire ‘git clone’ command. It’s the Ctrl+C and Ctrl+V party you always wanted to have! 🥳

So, the next time you’re browsing through GitHub and find a repository you’d like to clone, remember, you have a friend in the GitHub Clone Helper. It’s there to make sure that your ‘git clone’ days are behind you. Well, the typing part at least!

Oh, and let’s not forget about the big elephant in the room – Privacy. The GitHub Clone Helper is a true friend – it respects your privacy, doesn’t gossip (read: no unnecessary communication with servers), and doesn’t have a sneaky, sticky hand in your cookie jar (read: no collecting or sharing personal data). It’s all about that local life, just like your favorite artisanal bakery. 🍪🔒

If you’re a lazy smart developer like me, you’ll agree that there’s a certain beauty in not having to type ‘git clone’ before every URL. It’s not just about saving a few seconds; it’s about making our lives a little easier, a little more automated, and a lot more fun.

With the GitHub Clone Helper, we’re taking a step towards a world where ‘git clone’ is a thing of the past. A world where we can focus on the exciting stuff – code, coffee, and occasionally, sleep. 😴

So, get on board the GitHub Clone Helper train and let’s make a ‘git clone’ history! Choo-choo, fellow devs! 🚂💨

You can see these extension links below:

GitHub: https://github.com/flightlesstux/github-clone-helper 
Google Chrome Webstore: https://chrome.google.com/webstore/detail/kfabhahlbkfegapejbcigihbgjnpdobm 
Mozilla Firefox Add-ons: https://addons.mozilla.org/en-US/firefox/addon/github-clone-helper/

All feedback and stars are welcome! 🌟

Happy coding and may the force of the semicolon be with you! 💻🚀

---

# Unveiling the Mystery: Decoding IP Address Retrieval in Linux

URL: https://ercan.cloud/unveiling-the-mystery-decoding-ip-address-retrieval-in-linux/
Date: 2023-05-11T12:10:57+03:00

Understanding how to retrieve an IP address in Linux can be a bit tricky, especially for beginners. The good news? It's not as complicated...


Understanding how to retrieve an IP address in Linux can be a bit tricky, especially for beginners. The good news? It's not as complicated as it seems! We're here to simplify it for you. Let's dive in.

Getting Started

In a nutshell, an IP address is a unique identifier for your machine on a network. It's like your computer's postal address. In Linux, there are several ways to find this address, and two popular commands for doing this are ifconfig and ip.

Understanding the ifconfig Command

ifconfig is one of the oldest tools used to configure network interfaces in Linux. It displays the current network configuration and allows you to set up network interfaces.

Here's how to use it to find your IP address:

export LOCAL_IP=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 | awk '{print $2}')
echo $LOCAL_IP

Let's break this down:

ifconfig outputs the details of all network interfaces.

grep "inet " filters lines containing "inet", which precedes the IP address.

grep -Fv 127.0.0.1 filters out the localhost IP.

awk '{print $2}' prints the second column, which is the IP address.

Exploring the ip Command

The ip command is a newer tool that's intended to replace ifconfig. It provides more extensive features and is now the default command for network management in many Linux distributions.

Here's how to use it to find your IP address:

export LOCAL_IP=$(ip route get 1 | awk '{print $NF;exit}')
echo $LOCAL_IP

Breaking this down:

ip route get 1 finds the IP address that would be used to reach the Internet.

awk '{print $NF;exit}' prints the last field, which is the IP address.

Comparing ifconfig and ip

So, what's the difference between ifconfig and ip?

Age and Support: ifconfig is older and is being deprecated in many new Linux distributions in favor of ip. But, it's still available and used in many systems.

Functionality: ip offers more extensive functionality. It doesn't just handle network interfaces - it also manages routing tables, neighbors, and more.

Output: ip provides more detailed and precise output. The ifconfig command's output can be harder to parse and interpret, especially for complex setups.

Handling of Multiple Addresses: ifconfig can struggle with multiple IP addresses assigned to a single interface, whereas ip handles this smoothly.

Final Thoughts

While both ifconfig and ip can fetch your IP address, ip is a more powerful and versatile tool. But remember, these commands might output multiple IP addresses if you have more than one network interface.

Mastering these commands will give you better control over your Linux system and help you troubleshoot network issues more efficiently. Happy networking!

---

# The AWS Well-Architected Framework: A Key to Cloud Success

URL: https://ercan.cloud/the-aws-well-architected-framework-a-key-to-cloud-success/
Date: 2023-04-26T00:42:29+03:00

The cloud has revolutionized the way businesses operate, and Amazon Web Services (AWS) has consistently led the charge in cloud ...


Why It's Important, Why You Need It, and Its Key Role in Your Cloud Journey

The cloud has revolutionized the way businesses operate, and Amazon Web Services (AWS) has consistently led the charge in cloud computing. To help businesses build and maintain reliable, efficient, and secure applications in the cloud, AWS has introduced the Well-Architected Framework. In this blog post, we'll explore the importance of this framework, why you need to use it, and the key role it plays in your cloud journey.

What is the AWS Well-Architected Framework?

The AWS Well-Architected Framework is a set of best practices, design principles, and architectural recommendations for building and maintaining high-performing, resilient, and secure applications on the AWS cloud. It's designed to help you understand the potential trade-offs in your architecture and make informed decisions that can optimize your applications' performance, cost, and security.

Why is the AWS Well-Architected Framework Important?

Optimal Application Performance: The Well-Architected Framework helps ensure that your applications are designed to meet the demands of your users and provide a seamless experience. By adhering to the framework's best practices, you can optimize your applications to minimize latency, maximize throughput, and ensure consistent performance under different workloads.

Cost Efficiency: The framework's guidelines help you make cost-effective decisions for your infrastructure, ensuring you get the most out of your cloud investment. You'll learn how to leverage AWS services and features to optimize cost without compromising on performance or security.

Security and Compliance: The Well-Architected Framework provides guidance on implementing robust security measures for your applications and infrastructure. By following the recommended best practices, you can protect your data, ensure compliance with industry standards, and minimize the risk of security breaches.

Scalability and Resilience: The framework helps you design your applications to scale seamlessly with your business growth. It also offers insights into building fault-tolerant and resilient systems, ensuring that your applications remain available and functional even during unexpected events or failures.

Why Do You Need the AWS Well-Architected Framework?

Guided Approach: The Well-Architected Framework provides a structured and consistent approach to designing, deploying, and managing applications in the cloud. This allows you to build applications that are not only high-performing and secure but also cost-effective and scalable.

Continuous Improvement: The framework encourages a culture of continuous improvement, helping you to regularly review and enhance your applications to maintain optimal performance, security, and cost-efficiency.

Competitive Advantage: By adhering to the Well-Architected Framework, you can build applications that offer a superior user experience and outperform your competitors. This can help you gain a competitive edge in the market and drive business growth.

Faster Time to Market: The framework helps you to make informed decisions and avoid common pitfalls in the design and deployment of your applications, leading to a faster time-to-market and reduced risk of failure.

The Key Role of the AWS Well-Architected Framework

The AWS Well-Architected Framework plays a pivotal role in guiding businesses through the complex process of building and maintaining applications on the AWS cloud. It helps you achieve your desired outcomes by providing actionable guidance and best practices for designing, deploying, and managing your applications in the cloud. By using the framework, you can ensure that your applications are not only high-performing and secure but also cost-effective and scalable.

Conclusion

The AWS Well-Architected Framework is an invaluable tool for any business looking to build and maintain successful applications in the cloud. By following its guidance and best practices, you can optimize your applications

---

# Telnet Movies - A Nostalgic Look at the Dawn of Internet Movie Streaming

URL: https://ercan.cloud/telnet-movies-a-nostalgic-look-at-the-dawn-of-internet-movie-streaming/
Date: 2023-04-24T22:07:02+03:00

Long before Netflix, Hulu, and Amazon Prime Video transformed the way we consume content, there was a fascinating, lesser-known method...


Long before Netflix, Hulu, and Amazon Prime Video transformed the way we consume content, there was a fascinating, lesser-known method for watching movies online: Telnet movies. These text-based movies, transmitted over the Telnet protocol, were a stepping stone in the evolution of internet streaming. In this blog post, we will delve into the history of Telnet movies, their inventor, motivation behind the concept, and the first forum post or email related to them.

History

Telnet, short for "Teletype Network," is an application protocol used for communication between remote computers. It was developed in 1969 as part of the ARPANET (Advanced Research Projects Agency Network), which was the precursor to the modern Internet. Telnet enabled users to access and manage remote systems, making it an essential tool for early internet communication.

As the internet became more accessible in the early 1990s, users sought ways to share multimedia content online. This quest led to the birth of Telnet movies. These text-based animations were ASCII art representations of movies, painstakingly converted frame-by-frame into lines of text characters. These text-based movies were then transmitted via the Telnet protocol, allowing users to "stream" them in real-time on their computer terminals.

Invention and Motivation

The concept of Telnet movies can be attributed to a programmer and ASCII art enthusiast named Simon Jansen. In 1997, Jansen began working on a passion project: converting the entire 1977 Star Wars film into ASCII format. This ambitious undertaking required immense dedication, as Jansen manually converted each frame of the film into ASCII art. The project, dubbed "Star Wars ASCIIMATION," became an internet sensation, showcasing the potential of Telnet movies.

The motivation behind Telnet movies was to push the boundaries of internet capabilities and challenge the status quo. At the time, high-speed internet was still a luxury, and streaming video content was almost impossible. Telnet movies, with their low-bandwidth requirements, offered a unique way to experience multimedia content in a time when technology was still catching up.

The First Forum Post or Email: As the popularity of Telnet movies grew, discussion forums dedicated to the topic started to emerge. One of the earliest known forum posts discussing Telnet movies was made by Simon Jansen himself on the newsgroup "alt.ascii-art" in 1997. Jansen shared his Star Wars ASCIIMATION project with the community, sparking widespread interest in the concept.

List of Telnet Movies

Though the Star Wars ASCIIMATION remains the most iconic Telnet movie, a handful of other ASCII animations have since been created. 

Star Wars ASCIIMATION (1997) - The first Telnet movie, which covers the 1977 Star Wars film.

The Matrix (2000) - A text-based representation of the 1999 sci-fi classic.

Jurassic Park (2001) - An ASCII adaptation of the 1993 dinosaur blockbuster.

Pulp Fiction (2002) - A text-based retelling of Quentin Tarantino's 1994 crime drama.

2001: A Space Odyssey (2003) - An ASCII rendition of Stanley Kubrick's 1968 sci-fi masterpiece.

You can access the telnet movies and more tools via telnet telehack.com And also, if you install lolcat and run the command with lolcat like telnet telehack.com | lolcat you can colorize the movies and other outputs like below! : )

Telnet movies represent an important milestone in the history of internet streaming. They demonstrate the creativity and innovation of the early internet community and serve as a nostalgic reminder of how far technology has come. While modern streaming services have long since eclipsed Telnet movies in terms of convenience and accessibility, the ASCII animations will forever hold a special place in the hearts of those who experienced them in their heyday.

As the world of technology continues to advance, we can look back at Telnet movies with admiration for their ingenuity and the spirit of experimentation they represent. The creativity and determination shown by pioneers like Simon Jansen laid the groundwork for the streaming revolution that we enjoy today.

Although Telnet movies are now a niche interest, they continue to inspire a dedicated community of ASCII art enthusiasts and retro tech aficionados. For those interested in experiencing this unique form of entertainment, several Telnet movie servers still exist, allowing users to revisit these text-based animations.

In an era where high-definition video and immersive audio experiences have become the norm, Telnet movies serve as a humbling reminder of the early days of internet innovation. As we continue to push the boundaries of technology, it's important to remember and appreciate the pioneers who came before us, laying the foundation for the digital world we live in today.

So, the next time you enjoy a movie night streaming the latest blockbuster, take a moment to appreciate the journey that brought us here. From ASCII animations transmitted over Telnet to ultra-high-definition streaming, the evolution of internet movie streaming is a testament to human ingenuity and our relentless pursuit of progress.

I've inspired to write this article after read the Gizem's tweet on twitter. Thank you so much Gizem! 

hayırlı olsun ya internet bağlatmışım ama yine de çok güzel <3 pic.twitter.com/dW7EIZN7j6

&mdash; gizem (@gizemnkorkmaz) April 24, 2023

---

# Secure Your Media Files by Removing Metadata with AWS Lambda

URL: https://ercan.cloud/secure-your-media-files-by-removing-metadata-with-aws-lambda/
Date: 2023-04-20T15:36:52+03:00

Enhance privacy and security across various industries by stripping EXIF and other metadata from uploaded images and videos


In today's digital world, images and videos often contain metadata that reveals a surprising amount of information about the media file. This metadata, such as EXIF data in images, can include sensitive details like location, device information, and more. To protect user privacy and enhance security, businesses in various industries can benefit from removing this metadata from media files. In this blog post, we'll walk you through a simple AWS Lambda script that automatically removes metadata from uploaded images and videos in S3 buckets.

Industries That Can Benefit:

Social Media Platforms: Social media platforms handle a massive number of media uploads every day. By removing metadata from images and videos, these platforms can better protect user privacy and minimize the risk of unintentional information leaks.

E-Commerce: E-commerce websites often display user-generated content, such as product images and reviews. Stripping metadata from these media files ensures that customers' private information is not inadvertently exposed.

Healthcare: The healthcare industry deals with sensitive patient information, including images and videos from medical procedures. Removing metadata from these files is essential to comply with privacy regulations and protect patient confidentiality.

News and Media: Journalists and media organizations publish images and videos that may contain sensitive information about sources or locations. Stripping metadata can help protect this information and maintain the integrity of their reporting.

Education: Educational institutions often host and share various media files, such as lecture videos, research images, and student presentations. Removing metadata from these files ensures that private information about students, faculty, and research subjects is protected.

Benefits of Removing Metadata:

Enhanced Privacy: Stripping metadata from media files helps protect sensitive information about users, locations, and devices, safeguarding user privacy.

Security: By removing metadata, you reduce the risk of accidentally leaking sensitive information, which could be exploited by malicious actors.

Compliance: Removing metadata can help organizations comply with data protection regulations, such as GDPR or HIPAA, that require safeguarding user data.

Simplified Management: Automating metadata removal with AWS Lambda reduces the manual work needed to process media files, streamlining media management across your organization.

import boto3
import io
import os
from PIL import Image
from moviepy.editor import *

def lambda_handler(event, _):
    bucket_name = os.environ&#91;'S3_BUCKET_NAME']
    s3 = boto3.client('s3')
    object_name = event&#91;'Records']&#91;0]&#91;'s3']&#91;'object']&#91;'key']
    file_name, file_extension = os.path.splitext(object_name)
    
    supported_image_extensions = &#91;'.jpg', '.jpeg', '.png', '.tiff', '.tif', '.heic', '.heif']
    supported_video_extensions = &#91;'.mp4', '.mov', '.avi', '.mkv', '.webm']
    
    image_data = s3.get_object(Bucket=bucket_name, Key=object_name)

    if file_extension.lower() in supported_image_extensions:
        with io.BytesIO(image_data&#91;'Body'].read()) as image_file:
            image = Image.open(image_file)
            image_format = image.format
            
            with io.BytesIO() as new_image_data:
                image.save(new_image_data, format=image_format)
                new_image_data.seek(0)
                
                s3.put_object(Bucket=bucket_name, Key=object_name, Body=new_image_data, Tagging='ExifDeleted=True')

    elif file_extension.lower() in supported_video_extensions:
        with io.BytesIO(image_data&#91;'Body'].read()) as video_file:
            video = VideoFileClip(video_file)
            with io.BytesIO() as new_video_data:
                video.write_videofile(new_video_data, codec='libx264', audio_codec='aac')
                new_video_data.seek(0)
                
                s3.put_object(Bucket=bucket_name, Key=object_name, Body=new_video_data, Tagging='ExifDeleted=True')

Please note that the PIL and moviepy libraries are requires some shared libraries, which may not be available in the default Lambda environment. You'll need to create a custom Lambda layer that includes both shared libraries. You can follow the official guide to create a custom Lambda layer for FFmpeg.

Here is the Github Repository: https://github.com/flightlesstux/EXIF-Metadata-Remover

Conclusion

The AWS Lambda script we've provided makes it easy to remove metadata from images and videos uploaded to S3 buckets, enhancing privacy and security across a wide range of industries. By implementing this solution, you can protect user information, reduce potential risks, and ensure compliance with data protection regulations.

---

# CORS in Nginx: Configuration Guide for Enhanced Security

URL: https://ercan.cloud/cors-in-nginx-configuration-guide-for-enhanced-security/
Date: 2023-04-19T21:21:10+03:00

Cross-Origin Resource Sharing (CORS) is a crucial mechanism for enabling secure communication between web servers and browsers.


Cross-Origin Resource Sharing (CORS) is a crucial mechanism for enabling secure communication between web servers and browsers. By providing a way for web applications to request resources from different domains, CORS helps overcome the limitations of the Same-Origin Policy (SOP), which restricts web applications from accessing data on a different domain than the one serving the application.

In this blog post, we'll explore the importance of CORS, how it works, and how to configure it in Nginx for different scenarios. We'll also delve into key CORS concepts, such as allow-origin, allow-methods, allow-headers, and expose-headers.

CORS Importance

CORS is essential for modern web applications, as it enables communication between servers and clients from different domains. This ensures that applications can access data and services hosted on various domains securely while maintaining the integrity of the user's data.

Configuring CORS in Nginx

Nginx is a popular web server that can be easily configured to support CORS. In this section, we'll provide two different configuration examples for enabling CORS on an Nginx server. The first configuration will be for a single domain, while the second will cover multiple domains and subdomains.

Configuration Example 1: Single Domain (domain.com)

To enable CORS for a single domain, add the following to the server block in your Nginx configuration file:

location / {
    add_header 'Access-Control-Allow-Origin' 'https://domain.com';
    add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
    add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization';
    add_header 'Access-Control-Expose-Headers' 'Content-Length';
}

Configuration Example 2: Multiple Domains and Subdomains (domain.com, domain1.com, domain2.com, domain3.com, and subdomains)

To enable CORS for multiple domains and subdomains, use the following configuration in your Nginx server block:

location / {
    if ($http_origin ~* (https?://(?:.+\.)?(domain\.com|domain1\.com|domain2\.com|domain3\.com)$)) {
        add_header 'Access-Control-Allow-Origin' '$http_origin';
    }
    add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
    add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization';
    add_header 'Access-Control-Expose-Headers' 'Content-Length';
}

CORS Concepts

Access-Control-Allow-Origin: This header specifies which origins are allowed to access the resources on the server. In our examples, we either set a single domain or use a regular expression to match multiple domains and subdomains.

Access-Control-Allow-Methods: This header defines the allowed HTTP methods (e.g., GET, POST, OPTIONS) that the client can use when making requests to the server.

Access-Control-Allow-Headers: This header lists the allowed HTTP headers that the client can include in its requests. Common headers include "Content-Type" and "Authorization."

Access-Control-Expose-Headers: This header specifies which headers the server exposes to the client. In our examples, we expose the "Content-Length" header.

Conclusion

CORS is a vital security mechanism for modern web applications. By understanding and implementing CORS in Nginx, you can ensure that your web applications can securely access resources across different domains. With this guide, you should now be able to configure CORS in Nginx for single or multiple domains and subdomains while also understanding the importance of key CORS concepts like allow-origin, allow-methods, allow-headers, and expose-headers. Ensuring your Nginx server is CORS-compliant will improve the security and functionality of your web applications, providing a better user experience and increased compatibility across various platforms. By making your content SEO-friendly and easy to understand, you'll also improve your web application's discoverability and user engagement.

---

# List your S3 bucket objects easily with S3 Directory Listing

URL: https://ercan.cloud/list-your-s3-bucket-objects-easily-with-s3-directory-listing/
Date: 2023-04-18T15:35:21+03:00

Introducing S3-Directory-Listing, a simple and easy-to-use JS script and HTML combo that will turn your S3 bucket directory listing into ...


Are you tired of the boring, plain S3 bucket directory listing? Do you wish there was a way to make it more user-friendly and appealing? Well, look no further! Introducing S3-Directory-Listing, a simple and easy-to-use JS script and HTML combo that will turn your S3 bucket directory listing into a fancy, modern-looking file and folder listing with a nice UI and search functionality.

First things first, let's talk about usage. To use S3-Directory-Listing, all you need to do is clone the repository or copy the contents of dark-mode.css, s3.js and index.html to your S3 bucket. Then, update the bucketName variable in app.js with your bucket's name, and configure your S3 bucket settings. Access the index.html file in your browser, and voilà! You now have a fancy S3 directory listing. It's that simple!

But what about S3 bucket settings? To make this magic work, you need to configure your S3 bucket settings properly. First, make sure your S3 bucket is publicly accessible or at least accessible to the users you want to share the directory listing with. Next, set the bucket policy to allow public read access to your objects. Use the following policy, replacing <your-bucket-name> with your actual bucket name:

{
    "Version": "2012-10-17",
    "Statement": &#91;
        {
            "Sid": "S3DirectoryListing",
            "Effect": "Allow",
            "Principal": "*",
            "Action": &#91;
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": &#91;
                "arn:aws:s3:::<your-bucket-name>/*",
                "arn:aws:s3:::<your-bucket-name>"
            ]
        }
    ]
}

Replace <your-ip-address> with your desired IP address or a CIDR block to restrict access to specific IP addresses. Finally, enable static website hosting for your bucket, and note down the bucket's static website endpoint (e.g., http://<your-bucket-name>.s3-website-<your-region>.amazonaws.com/index.html or https://<your-bucket-name>.s3.amazonaws.com/index.html for secure connection).

Now that your S3 bucket settings are configured, you can enjoy your brand new, fancy-pants S3 directory listing! But wait, there's more. You can customize the number of items shown per page by modifying the itemsPerPage variable in s3.js.

Update your Cross-origin resource sharing (CORS) settings like below.

&#91;
    {
        "AllowedHeaders": &#91;
            "*"
        ],
        "AllowedMethods": &#91;
            "GET"
        ],
        "AllowedOrigins": &#91;
            "*"
        ],
        "ExposeHeaders": &#91;]
    }
]

S3-Directory-Listing is not only easy to use, but it's also highly customizable. You can tweak the script and HTML to fit your specific needs and preferences. And if you run into any issues or have any questions, you can head over to the GitHub repo and let the creators know. They'd love to hear from you!

In conclusion, S3-Directory-Listing is a game-changer for anyone who wants to fancy up their S3 bucket directory listing. It's easy to use, highly customizable, and best of all, it's free! So why settle for a boring, plain S3 directory listing when you can have a modern, user-friendly one? Give S3-Directory-Listing a try today and see for yourself how much of a difference it can make. 

You can also use in Dark Mode! Here is the demo: https://s3-directory-listing.s3.amazonaws.com/index.html

You can find the well documented README and the source code is Github Repository: https://github.com/flightlesstux/S3-Directory-Listing

Happy listing!

---

# Keeping Composer Packages Up-to-Date with Composer Guardian: Why It's Essential

URL: https://ercan.cloud/keeping-composer-packages-up-to-date-with-composer-guardian-why-its-essential/
Date: 2023-04-15T19:09:12+03:00


As a PHP developer, chances are you're already familiar with Composer, the dependency manager for PHP. It's an essential tool for managing packages, streamlining updates, and ensuring that your projects run smoothly. One crucial aspect of using Composer is keeping your packages up-to-date. In this blog post, we will discuss the importance of keeping your Composer packages current and how Composer Guardian, an open-source script, can help you achieve that.

Why Keep Composer Packages Up-to-Date?

Security: Outdated packages can expose your application to security risks, as they may contain vulnerabilities that have been addressed in newer versions. By keeping your packages up-to-date, you minimize the risk of being targeted by hackers.

Performance: Newer versions of packages often contain performance improvements and optimizations that can lead to a faster, more efficient application. Staying up-to-date ensures that you're using the most efficient version of a package.

Compatibility: As PHP and other packages evolve, compatibility issues can arise. By regularly updating your packages, you minimize the risk of encountering conflicts or issues related to deprecated functionality.

Bug Fixes: Package updates often contain bug fixes that can resolve issues you may be experiencing in your application. By staying current, you can avoid potential problems and ensure a smoother development process.

New Features: Updated packages often introduce new features that can benefit your application. By keeping your packages current, you can take advantage of these features and continue to innovate within your projects.

Introducing the Composer Guardian

The provided script, called Composer Guardian, is designed to help you stay on top of package updates. It reads your composer.json file, checks for updates, and generates a report detailing which packages need to be updated.

Key Features:

Fetches the latest stable version of each package listed in your composer.json file

Skips packages with certain prefixes (e.g., 'ext-') or excluded packages (e.g., 'php')

Displays a table with the package name, current version, and latest version

Optionally sends the report to a Slack channel using a webhook URL

Usage:

To use Composer Guardian, simply clone the GitHub repository and run the script with the appropriate command-line options or environment variables. For example:

python/python3 composer_guardian.py --composer-file-path /path/to/composer.json --slack-webhook-url https://hooks.slack.com/services/...

Or, using environment variables:

export COMPOSER_FILE_PATH=/path/to/composer.json
export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
python/python3 composer_guardian.py

Automate the check: CI/CD on a schedule

Running the script by hand works exactly until the week you forget. The check only earns its keep when it runs on a schedule, without anyone remembering it. Composer Guardian exits with a non-zero status code when updates are available, so any CI system can treat outdated dependencies as a first-class result: report it, or fail the pipeline outright.

With GitHub Actions, a weekly cron plus a manual trigger is enough. Copy the workflow into your PHP project as .github/workflows/composer-guardian.yml and add your Slack webhook URL as a repository secret named SLACK_WEBHOOK_URL. The workflow runs on a weekly cron ("0 9 * * 1", every Monday at 09:00 UTC) plus workflow_dispatch, sets up Python, clones the Composer Guardian repository, and runs composer_guardian.py against the project's composer.json with the Slack webhook from the secret.

The trailing || true is the one decision you have to make. Keep it and the job always goes green: the report lands in Slack and nobody's deploy is blocked by a package that shipped a patch release overnight. Remove it and outdated dependencies fail the pipeline, which is the right call for teams that treat dependency drift as a build error rather than a notification. Neither default is correct for everyone; pick the failure mode you can live with.

On GitLab, the same pattern is a scheduled pipeline: the repository ships a ready-to-use job in examples/gitlab-ci.yml, wired to run only when triggered by a pipeline schedule. Both files live in the examples/ directory of the repository, ready to copy into your project.

Conclusion

Keeping your Composer packages up-to-date is crucial for maintaining the security, performance, and overall health of your PHP applications. Composer Guardian provides an easy-to-use solution for staying current with package updates and can help you streamline your development process. Give it a try and see how it can benefit your projects. Don't forget to star the repository if you find it helpful, and feel free to contribute or open issues if you encounter any problems.

---

# What is Carrier Gateway on AWS?

URL: https://ercan.cloud/what-is-carrier-gateway-on-aws/
Date: 2023-02-24T20:19:15+03:00

Amazon Web Services (AWS) offers many services to provide a seamless and secure cloud computing experience to its users. One of these ...


Amazon Web Services (AWS) offers many services to provide a seamless and secure cloud computing experience to its users. One of these services is Carrier Gateway, which is designed to provide a simplified and centralized solution for connecting customer networks to AWS.

In this article, we will discuss the concept of Carrier Gateway on AWS and its benefits in detail.

What is Carrier Gateway on AWS?

Carrier Gateway is a service that enables AWS customers to establish a direct and private connection between their on-premises data center or network and their Virtual Private Cloud (VPC) on AWS. It allows users to extend their network topology to AWS, thus providing a secure and high-bandwidth connection between their on-premises infrastructure and AWS.

Carrier Gateway provides a single gateway to connect to multiple AWS Regions, thus eliminating the need to create multiple VPN connections for each region. This makes it easier for customers to manage their network infrastructure and reduce operational complexity.

Benefits of Carrier Gateway on AWS:

Enhanced Security Carrier Gateway provides a secure and private connection between customer networks and AWS. It uses industry-standard encryption and authentication protocols to ensure data privacy and security. This eliminates the need to use public internet connections and reduces the risk of cyberattacks and data breaches.

  High-Performance Carrier Gateway provides high-bandwidth connectivity between customer networks and AWS. This ensures faster data transfer and reduced latency, thus enhancing application performance and user experience.

  Simplified Network Management Carrier Gateway provides a centralized solution for connecting customer networks to AWS. This eliminates the need to create multiple VPN connections for each region, thus reducing operational complexity and network management overheads.

  Cost-Effective Carrier Gateway provides a cost-effective solution for connecting customer networks to AWS. It eliminates the need to use expensive leased lines or MPLS circuits, thus reducing connectivity costs significantly.

  

How Carrier Gateway Works

Carrier Gateway works by establishing a direct and private connection between customer networks and AWS. It provides a Layer 2 Ethernet interface that connects to the customer's on-premises router or switch. The Carrier Gateway interface is connected to an AWS Direct Connect Gateway, which in turn connects to the VPC on AWS.

The Carrier Gateway interface can be configured with VLAN tagging to support multiple VPCs. This allows customers to extend their network topology to multiple VPCs on AWS using a single Carrier Gateway.

Carrier Gateway supports Border Gateway Protocol (BGP) routing to enable dynamic routing between the customer network and the VPC on AWS. This ensures that traffic is always routed over the most optimal path, thus providing high performance and reliability.

Conclusion

Carrier Gateway on AWS is a powerful service that provides a secure, high-bandwidth, and cost-effective solution for connecting customer networks to AWS. It eliminates the need to use public internet connections, simplifies network management, and enhances application performance.

If you are considering connecting your on-premises infrastructure to AWS, Carrier Gateway is an excellent option to consider. It provides a seamless and secure connection that can help you reduce costs, enhance security, and simplify network management.

---

# Creating SSH Keys for Secure Access to AWS EC2 Instances with Terraform

URL: https://ercan.cloud/creating-ssh-keys-for-secure-access-to-aws-ec2-instances-with-terraform/
Date: 2023-02-15T09:11:24+03:00

When working with cloud infrastructure, security is of utmost importance. One critical aspect of security is controlling access to cloud ...


When working with cloud infrastructure, security is of utmost importance. One critical aspect of security is controlling access to cloud resources, and this is particularly important when working with AWS EC2 instances. SSH keys are one way to secure access to EC2 instances, and with Terraform, it is straightforward to create and manage these keys.

SSH keys are used for secure access to an EC2 instance, and they consist of two parts - a public key and a private key. The public key is uploaded to the EC2 instance, and the private key is used to authenticate with the instance. The private key should be kept secure and not shared with anyone else.

Terraform is a powerful tool for managing infrastructure as code, and it can be used to create and manage SSH keys for secure access to AWS EC2 instances. Here's how to do it.

First, create an SSH key pair. This can be done with the ssh-keygen command on your local machine. To generate an RSA key pair with a 4096-bit key size, run the following command:

ssh-keygen -t rsa -b 4096

This will generate a public key and a private key in the .ssh directory in your home directory. The public key has a .pub extension and should be uploaded to the EC2 instance.

Next, create a Terraform module to manage the SSH key. The module should include the following code:

resource "aws_key_pair" "ssh_key" {
  key_name   = "ssh_key"
  public_key = file("~/.ssh/id_rsa.pub")
}

This creates an AWS key pair resource with the name "ssh_key" and uploads the public key from the local machine to AWS. The file() function is used to read the contents of the public key file on the local machine.

The next step is to reference the SSH key in the EC2 instance resource. This can be done with the key_name parameter in the aws_instance resource:

resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
  key_name      = aws_key_pair.ssh_key.key_name

  // other instance configuration
}

This creates an EC2 instance with the specified AMI and instance type, and associates the SSH key with the instance using the key_name parameter. The value of aws_key_pair.ssh_key.key_name is a reference to the name of the key pair resource created in the previous step.

Finally, initialize the Terraform module with terraform init, and apply the changes with terraform apply. Terraform will create the SSH key pair resource and the EC2 instance resource with the associated SSH key.

With this setup, you can now securely access the EC2 instance using the private key generated earlier. To do this, use the ssh command on your local machine:

ssh -i ~/.ssh/id_rsa ec2-user@<instance_public_ip>

This will connect to the EC2 instance using the private key and the default user ec2-user. Replace <instance_public_ip> with the public IP address of the EC2 instance.

In summary, using SSH keys for secure access to AWS EC2 instances is a crucial part of cloud infrastructure security. With Terraform, it is straightforward to create and manage these keys as part of your infrastructure as code. By following the steps outlined in this article, you can ensure secure access to your EC2 instances and protect your cloud infrastructure.

---

# What is an Egress only internet gateways in AWS?

URL: https://ercan.cloud/what-is-an-egress-only-internet-gateways-in-aws/
Date: 2023-01-31T06:48:25+03:00

Within a VPC, an Egress-Only Internet Gateway is an essential component that enables outbound traffic from the VPC to the Internet.


Amazon Web Services (AWS) is one of the leading cloud computing platforms, providing a variety of infrastructure services to businesses of all sizes. One of the essential components of AWS is Virtual Private Cloud (VPC), which allows users to isolate their resources in a logically isolated virtual network. Within a VPC, an Egress-Only Internet Gateway is an essential component that enables outbound traffic from the VPC to the Internet.

An Egress-Only Internet Gateway is a horizontally scaled, redundant, and highly available VPC component that provides a secure way for outbound-only Internet traffic from instances in a VPC to flow to the Internet. It operates as a stateful gateway, which means that it keeps track of the network connections initiated from instances in the VPC, and it automatically allows the corresponding return traffic to flow back into the VPC.

The main advantage of using an Egress-Only Internet Gateway is that it provides a secure and reliable way for instances in a VPC to access the Internet. For example, it can be used to allow instances in a VPC to access public AWS services such as Amazon S3 or Amazon EC2, or to access public websites, such as www.google.com. An Egress-Only Internet Gateway also enhances the security of instances in a VPC, as it blocks all incoming traffic from the Internet to instances in the VPC, and only allows outgoing traffic.

To use an Egress-Only Internet Gateway, you must create a VPC, configure a route table for your VPC, and associate the route table with the VPC. Then, you need to create an Egress-Only Internet Gateway and associate it with the VPC. You can also associate multiple subnets in a VPC with an Egress-Only Internet Gateway, which provides more flexibility and security.

One of the benefits of using an Egress-Only Internet Gateway is that it enables you to have full control over the outbound Internet traffic from instances in your VPC. For example, you can control access to specific websites or services, and monitor the traffic for compliance and security purposes. Additionally, you can use AWS Network Firewall or Amazon VPC Security Groups to control the outbound traffic from instances in your VPC.

Another benefit of using an Egress-Only Internet Gateway is that it helps reduce the operational overhead and complexity of managing Internet access for instances in your VPC. For example, you do not need to configure a NAT gateway or a VPN connection to the Internet, which can be challenging and time-consuming. Instead, you can simply create an Egress-Only Internet Gateway, associate it with your VPC, and configure the route table, which can be done quickly and easily.

Conclusion

In conclusion, an Egress-Only Internet Gateway is an essential component of AWS Virtual Private Cloud (VPC) that provides a secure and reliable way for instances in a VPC to access the Internet. It provides full control over the outbound Internet traffic from instances in your VPC, enhances security, and reduces operational overhead. By using an Egress-Only Internet Gateway, businesses can securely access public AWS services and the Internet, without having to worry about managing the underlying infrastructure.

---

# What is an Internet Gateway in AWS?

URL: https://ercan.cloud/what-is-an-internet-gateway-in-aws/
Date: 2023-01-11T14:50:24+03:00


Amazon Web Services (AWS) Internet Gateway is a horizontally scalable, redundant, and highly available VPC component that allows communication between instances in your VPC and the Internet. It, therefore, acts as a bridge between your VPC and the Internet.

An Internet Gateway is not automatically created when you create a VPC. Instead, you must create and attach an Internet Gateway to your VPC before your instances in the VPC can communicate with the Internet. Once an Internet Gateway is attached to a VPC, you can then route Internet traffic to your instances.

Internet Gateways are horizontally scaleable, meaning that they automatically adjust their capacity to handle the number of requests that pass through them. They are also redundant, which means that if one Internet Gateway fails, another one takes over and ensures that there is no interruption in service. This redundancy ensures that your instances are always able to communicate with the Internet.

To create an Internet Gateway, you must first log in to the AWS Management Console and navigate to the VPC dashboard. Once there, you can click on the "Internet Gateways" menu and select "Create Internet Gateway." You will then be prompted to give your Internet Gateway a name and select the VPC that it should be attached. Once the Internet Gateway is created, it will be in a "detached" state. To attach the Internet Gateway to a VPC, you must click on the Internet Gateway and select "Actions," then "Attach to VPC."

Once an Internet Gateway is attached to a VPC, it can be used to route Internet traffic to instances in that VPC. To do this, you must create a route in the VPC's routing table that points Internet traffic (0.0.0.0/0) to the Internet Gateway. This is often referred to as the "default route" for Internet traffic.

When you create a new VPC, it automatically comes with a default route table, which you can use to route Internet traffic to the Internet Gateway. You can also create additional route tables and associate them with specific subnets in your VPC. This allows you to control the Internet access for instances in those subnets.

AWS Internet Gateway allows access to Internet resources over IPv4 and IPv6 protocols. Additionally, if you want to connect to your on-premises resources, you can create a VPN connection or a Direct Connect link between your on-premises data center and your VPC over Internet Gateway. It also provides network address translation (NAT) service, which allows instances in a private subnet to connect to the Internet, but prevents the Internet from initiating connections with those instances.

AWS Internet Gateway also provides security features such as Security Group and Network Access Control List (NACL) to control inbound and outbound traffic to your instances via Internet Gateway. It allows you to create rules that permit or deny traffic based on protocol, port, and source or destination IP address. You can also use AWS Web Application Firewall (WAF) to protect your applications and APIs from various web-based attacks.

Conclusion

In summary, Amazon Web Services (AWS) Internet Gateway is a horizontally scalable, redundant, and highly available VPC component that enables the communication between instances in a VPC and the Internet. It acts as a bridge between a VPC and the Internet and allows you to route Internet traffic to your instances. Internet Gateways also provide security features such as security groups and network access control lists to control inbound and outbound traffic, as well as a way to connect to on-premises resources via VPN or Direct Connect.

---

# What are Route Tables on AWS VPC?

URL: https://ercan.cloud/what-are-route-tables-on-aws-vpc/
Date: 2023-01-03T09:03:38+03:00


Route tables in Amazon Web Services (AWS) are used to determine how traffic is directed within a Virtual Private Cloud (VPC). They contain a set of rules, known as routes, that specify which network traffic is directed to which network interface.

Each subnet in a VPC must be associated with a route table, which controls the traffic for that subnet. A subnet can only be associated with one route table at a time, but a route table can be associated with multiple subnets. This allows you to have different routing rules for different subnets within a VPC.

A route table consists of a set of rules, called routes, that specify which traffic is directed to which network interface. A route consists of a destination and a target. The destination is the IP range that the traffic is being sent to, and the target is the network interface to which the traffic is being sent to.

There are two types of routes in a route table:

Local routes: These routes apply to traffic within the VPC. Every VPC has a default local route that allows traffic to flow between subnets within the VPC.

Internet Gateway routes: These routes apply to traffic going to and from the internet. Every VPC has a default Internet Gateway route that allows traffic to flow between the VPC and the internet.

In addition to the default route table, you can create additional custom route tables. This allows you to specify custom routing rules for specific subnets within your VPC. For example, you might want to create a custom route table for a subnet that is hosting a database, to ensure that traffic to and from the database is routed properly.

When you create a new VPC, AWS automatically creates a default route table. This default route table contains a local route and an Internet Gateway route, as described above. You can modify the routes in the default route table, but you cannot delete them.

It is important to properly configure your route tables in order to ensure that traffic is directed correctly within your VPC. Incorrectly configured route tables can lead to network connectivity issues and negatively impact your applications' performance.

There are a few key things to consider when configuring your route tables:

Make sure that you have an Internet Gateway route for any subnet that needs to access the internet.

Make sure that you have local routes for any subnets that need to communicate with each other within the VPC.

Make sure that you have a default route table, and that it is properly configured.

Consider creating custom route tables for specific subnets that have unique routing requirements.

Conclusion

In summary, route tables in AWS are used to control the routing of traffic within a VPC. They consist of a set of routes that specify which traffic is directed to which network interface, and they are associated with individual subnets within a VPC. Properly configuring your route tables is important for ensuring that traffic is directed correctly within your VPC and for maintaining the performance of your applications.

---

# What is Subnet on AWS VPC?

URL: https://ercan.cloud/what-is-subnet-on-aws-vpc/
Date: 2022-12-10T09:05:25+03:00

Subnets in AWS VPC are used to divide a VPC into multiple logical networks. This allows users to isolate resources and control access to ...


Amazon Web Services (AWS) Virtual Private Cloud (VPC) is a cloud computing service that allows users to create and manage their own virtual network in the cloud. This allows users to have complete control over their network and resources, including the ability to customize subnetting for their VPC.

Subnets in AWS VPC are used to divide a VPC into multiple logical networks. This allows users to isolate resources and control access to those resources. Subnets can be either public or private, with public subnets having access to the internet and private subnets not having access to the internet.

Creating subnets in AWS VPC is easy and can be done through the AWS Management Console. First, the user must select their VPC and then click on the "Subnets" tab. From there, the user can select "Create Subnet" and enter the name, VPC, and availability zone for the subnet. They can also specify whether the subnet is public or private.

Once the subnet is created, the user can then add resources to the subnet, such as EC2 instances or RDS databases. The user can also control access to the subnet through the use of security groups and network ACLs.

One important consideration when creating subnets in AWS VPC is the CIDR block. CIDR (Classless Inter-Domain Routing) blocks are used to specify the range of IP addresses that can be used in a subnet. The user must specify a CIDR block when creating a subnet, and the CIDR block must be within the VPC's CIDR block range.

Another important consideration is the availability zone for the subnet. AWS VPC allows users to create subnets in multiple availability zones within a region. This provides high availability and redundancy for the subnet and its resources.

Subnetting can also be useful for segmenting a VPC into different environments, such as development, staging, and production. This allows users to isolate resources and control access between environments.

There are also additional benefits to using subnets in AWS VPC. For example, users can create a NAT gateway in a public subnet and use it to enable internet access for private subnets. This allows resources in the private subnet to access the internet without having a public IP address.

Users can also use VPC peering to connect two VPCs together and share resources between them. This can be useful for creating a more complex network architecture, such as connecting a VPC in one region to a VPC in another region.

Conclusion

Subnets in AWS VPC provide users with the ability to customize and control their virtual network in the cloud. Subnetting allows users to segment their VPC into multiple logical networks, isolate resources, and control access to those resources. It also provides benefits such as high availability and redundancy, and the ability to connect VPCs together. Overall, using subnets in AWS VPC can greatly enhance the flexibility and functionality of a cloud-based network.

---

# What is AWS VPC?

URL: https://ercan.cloud/what-is-aws-vpc/
Date: 2022-12-07T05:56:09+03:00

Learn about Amazon Web Services (AWS) Virtual Private Cloud (VPC), a service that allows users to create and manage their own virtual pri...


What is AWS VPC?

Amazon Web Services (AWS) Virtual Private Cloud (VPC) is a service that allows users to create and manage their own virtual private network (VPN) in the AWS cloud. This allows users to have complete control over their own network environment, including the ability to select their own IP address range, create subnets, and configure route tables and network gateways.

One of the main benefits of using VPC is the ability to securely connect to the internet and to other AWS services. With VPC, users can create a public-facing subnet for their web servers, and a private-facing subnet for their databases. This allows for better security and isolation of sensitive data.

VPC also allows for greater control over network access. Users can use security groups and network ACLs to control inbound and outbound traffic at the subnet and network interface levels. This allows users to create granular rules to restrict access to specific IP addresses or ranges, ports, and protocols.

Another benefit of VPC is the ability to use AWS PrivateLink. This allows users to securely access services over the AWS network, without the need for an internet gateway or a VPN connection. This can improve security and reduce data transfer costs.

VPC also allows for seamless integration with other AWS services. For example, users can use VPC to connect to Amazon Elastic Compute Cloud (EC2) instances, Amazon Relational Database Service (RDS) databases, and other AWS services. This allows for easy deployment and management of complex, multi-tier applications in the AWS cloud.

In addition to these benefits, VPC also offers several other features and capabilities. For example, users can use VPC Flow Logs to capture information about the IP traffic going to and from their network interfaces. This can be useful for monitoring and troubleshooting network issues.

VPC also allows for the creation of VPN connections to on-premises networks. This can be useful for organizations that have a hybrid cloud environment, or for those that want to connect their AWS resources to their existing IT infrastructure.

Conclusion

Overall, AWS VPC provides users with a highly customizable and secure network environment in the AWS cloud. With VPC, users can easily connect to the internet and other AWS services, control network access, and integrate with other AWS services. This allows for greater flexibility and control over their cloud environment, enabling them to build and deploy complex, multi-tier applications in the AWS cloud.

---

# Prevent nginx from caching DNS for proxy

URL: https://ercan.cloud/prevent-nginx-from-caching-dns-for-proxy/
Date: 2022-08-22T12:17:44+03:00

If you are having problems with nginx proxy DNS caching, you can see the config for flush and/or clear DNS caching continuously...


Hello everyone,

If you are using nginx as a proxy and/or reverse proxy, the nginx is caching the DNS information and if you are using AWS Application Load Balancer behind the nginx, and nginx sometimes needs to restart and/or DNS flushing to send a request to the AWS Application Load Balancer because AWS is always giving a CNAME and changing the Load Balancer IP address frequently. 

There is no option to flush DNS on nginx, the only option is to restart the nginx and you can't solve the problems with restart always : ) You can use the nginx config below to fix this DNS Caching problem.

server {
   listen       80;
   server_name  proxy-request.ercanermis.net;

   include /etc/nginx/default.d/*.conf;

   set $proxydestination "backend.ercanermis.com";
   proxy_pass https://$proxydestination;
   proxy_http_version 1.1;
   proxy_set_header Host proxy-request.ercanermis.net;
   proxy_redirect off;
   proxy_cache_bypass 1;
   proxy_no_cache 1;
   resolver 8.8.8.8 valid=5s;
   }
}

If you have any questions, please feel free to comment, thank you!

---

# Scaling PHP Applications on AWS

URL: https://ercan.cloud/scaling-php-applications-on-aws/
Date: 2022-08-10T05:18:09+03:00

You can understand the running highly available, performant, and secure PHP applications on AWS with easy 14 steps...


You can understand the running highly available, performant, and secure PHP applications on AWS.

Amazon Route 53 routes end-user requests resolving Domain Name Service (DNS).

Amazon CloudFront caches content and accelerates delivery, leveraging global points of presence. CloudFront also handles SSL ending, integrating with Amazon Certificate Manager, which automatically creates and renews SSL certificates at no cost.

AWS Web Application Firewall integration with CloudFront and Application Load Balancer mitigates OWASP's top 10 application vulnerabilities.

The Application Load Balancer routes HTTP/S requests to EC2 instances running on private subnets.

An Amazon Linux 2 AMI contains the PHP and other needed binaries, such as the AWS SDK for PHP.

The Amazon CloudWatch Agent installed on the Amazon Linux 2 AMI streams application logs, additional host-level metrics, and custom business metrics.

Amazon EC2 Auto Scaling manages the instance launch based on metrics such as CPU and memory. It uses Amazon Graviton instances for cost optimization.

Using Systems Manager Session Manager, connect to EC2
instances with web-based sessions on the AWS Console. There is no need for key pairs of SSH ports to be open.

Database credentials are securely stored on AWS Secrets Manager. Using the AWS SDK for PHP, the application code retrieves the credentials stored on Secrets Manager though
an IAM Role.

Application code is safely stored on AWS CodeCommit using the familiar Git command line interface (CLI).

AWS CodePipeline implements continuous integration and continuous deployment (CI/CD), orchestrating code deployment using an AWS CodeDeploy hook that triggers
when new EC2 instances are launched.

Amazon ElastiCache for Redis caches session data.

Amazon Aurora Multi-AZ enables high availability. The application connects via DNS endpoint that handles failover automatically in case of failure. The Aurora reader endpoint handles read operations, offloading the Aurora writer instance load.

Amazon Elastic File System (Amazon EFS) stores and
shares web content with the Auto Scaling group.

---

# Create an S3 Bucket and Set a Policy via CLI

URL: https://ercan.cloud/create-an-s3-bucket-and-set-a-policy-via-cli/
Date: 2022-07-17T12:09:37+03:00

I really like to use CLI commands and it's my daily routine. Today, I'll tell to you "How do you create an S3 Bucket on AWS" and "Put an ...


I really like to use CLI commands and it's my daily routine. Today, I'll tell to you "How do you create an S3 Bucket on AWS" and "Put an S3 Bucket Policy" via CLI. Let's start...

The first thing, you should set your AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY. It means you should run aws configure the command before. If you do this, you can continue.

Check the S3 Buckets First

First of all, we need to make sure which buckets we have. When I run the aws s3 ls command, it will return the existing buckets.

➜  ~ aws s3 ls
2021-04-26 11:01:32 ercanermis
2021-05-20 19:14:04 ercanermis-aws-ssm-logs
2021-04-26 11:01:43 ercanermis-com
2021-04-26 11:02:08 www-ercanermis-com

Create an S3 Bucket

S3 Bucket creating command is also simple and easy. It's aws s3 mb. Let's do it and list buckets again.

➜  ~ aws s3 mb s3://hello-this-is-new-bucket
make_bucket: hello-this-is-new-bucket

Now, I want to list all existing buckets again to make sure.

➜  ~ aws s3 ls
2021-04-26 11:01:32 ercanermis
2021-05-20 19:14:04 ercanermis-aws-ssm-logs
2021-04-26 11:01:43 ercanermis-com
2022-07-17 14:23:29 hello-this-is-new-bucket
2021-04-26 11:02:08 www-ercanermis-com

Yes, I see my newly created bucket. The bucket name is hello-this-is-new-bucket.

Check a Bucket Policy on S3 Bucket

We created a bucket in the previous step but what about bucket policy? First of all, we need to check is there any policy existing or not. Here is an example;

➜  ~ aws s3api get-bucket-policy --bucket hello-this-is-new-bucket

An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

The freshly created bucket has no policy on default. Now, time to create a policy with CLI.

Create a Bucket Policy

The policy should be in JSON format and you can create a JSON file. My bucket policy allows all users to retrieve any object  hello-this-is-new-bucket except those in the secret-folder. It also grants put and delete permission to the "root" user of the AWS account "1234-5678-9012".

my-s3-bucket-policy.json:
{
   "Statement": &#91;
      {
         "Effect": "Allow",
         "Principal": "*",
         "Action": "s3:GetObject",
         "Resource": "arn:aws:s3:::hello-this-is-new-bucket/*"
      },
      {
         "Effect": "Deny",
         "Principal": "*",
         "Action": "s3:GetObject",
         "Resource": "arn:aws:s3:::hello-this-is-new-bucket/secret-folder/*"
      },
      {
         "Effect": "Allow",
         "Principal": {
            "AWS": "arn:aws:iam::123456789012:root"
         },
         "Action": &#91;
            "s3:DeleteObject",
            "s3:PutObject"
         ],
         "Resource": "arn:aws:s3:::hello-this-is-new-bucket/*"
      }
   ]
}

Put a Bucket Policy on S3 Bucket

Putting a policy is also easy. You can see it below.

aws s3api put-bucket-policy --bucket hello-this-is-new-bucket --policy file://my-s3-bucket-policy.json

After the put-bucket-policy command, the aws cli doesn't give any output but you can also know how you can check your bucket policy via CLI and you can check if you want to make sure.

Thank you so much for reading this article, happy coding!

---

# Issue a Let's Encrypt SSL with the AWS Route53

URL: https://ercan.cloud/issue-a-lets-encrypt-ssl-with-the-aws-route53/
Date: 2022-07-16T04:47:34+03:00

In this article, I want to tell you how you can issue a Let's Encrypt SSL with the AWS Route53 service. Let's start to learn.


Hello,

In this article, I want to tell you how you can issue a Let's Encrypt SSL with the AWS Route53 service. Let's start to learn.

What is Let's Encrypt?

Let's Encrypt is a non-profit certificate authority run by Internet Security Research Group (ISRG) that provides X.509 certificates for Transport Layer Security encryption at no charge. It means, you can get a FREE SSL certificate for your project/app and it the SSL certificate will be valid for 90 days. You need to make some automation to renew automatically.

What is AWS Route53?

Amazon Route 53 is a scalable and highly available Domain Name System service. Released on December 5, 2010, it is part of Amazon.com's cloud computing platform, Amazon Web Services. You can register a new domain or transfer your existing domain or just use the DNS management system for your domain.

For this article, I'll use one of my domain which is registered on Google Domains and I decided to just use the DNS management system. Just in case, the domain type should be a public hosted zone on Route53. My domain is "ermis.email" You can feel free to check NS records for my domain : )

Why I'm using Route 53 to issue a Let's Encrypt SSL?

When you request a new SSL certificate from Let's Encrypt, the certbot sends a request to the Let's Encrypt servers and I don't want to send and receive an HTTP (insecure) request out of AWS to get an encryption certificate : ) I'm thinking, it's a little bit weird and may be made some security problems. 

Otherwise, maybe you just need an SSL certificate and you don't want to bind your SSL certificate directly to an nginx or apache or any kind of web server service. And also, it's more convenient and fast and able to make some automation this way...

IAM Policy

I decided to use an AWS EC2 to issue an SSL directly in AWS and first of all, we need to create an IAM Policy for this operation to access Route53. My EC2 has an Elastic IP to make more secure the IAM Policy and restrict access to Route53 resources. You can't use this policy as an IAM Role and I configure it for aws-cli operations with a specific profile name "certbot-route53".  Here is an example IAM Policy below;

{
    "Version": "2012-10-17",
    "Statement": &#91;
        {
            "Sid": "CertbotRoute53",
            "Effect": "Allow",
            "Action": &#91;
                "route53:GetChange",
                "route53:ListHostedZones"
            ],
            "Resource": "*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "3.92.136.59/32"
                }
            }
        },
        {
            "Sid": "CertbotRoute531",
            "Effect": "Allow",
            "Action": "route53:ChangeResourceRecordSets",
            "Resource": "arn:aws:route53:::hostedzone/Z01859902ZBFBUXJ0AV3J",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "3.92.136.59/32"
                }
            }
        }
    ]
}

My mail server's DNS name is "hermes.ermis.email" and A record is pointing to 3.65.142.23 with Simple Routing. I'll request a Let's Encrypt SSL for "hermes.ermis.email" My domain TLD is not com/net/org maybe looks different but it's not a problem. It's just domain and DNS : )

Requirements for EC2

If you are running on freshly launched with an official Amazon Linux 2 AMI you should install some prerequisites. First of all, we need to install an epel repository to install certbot and certbot-dns-route53 plugin. Let's do it with a single line command!

&#91;root@ip-172-31-89-11 ~]# amazon-linux-extras install -y epel && yum install -y certbot python2-certbot-dns-route53

Let's Issue!

You can check my issuing command below.

&#91;root@ip-172-31-89-11 ~]# export AWS_PROFILE=certbot-route53; \
> certbot certonly --dns-route53 \
> --agree-tos \
> -m hello@ermis.email \
> --non-interactive \
> -d hermes.ermis.email

One Last Thing

Don't forget to set a cron to check SSL need a renewal. You can also use the command into your cron.

00 00 * * * export AWS_PROFILE=certbot-route53; certbot renew -q

Thank you for reading my article, I hope you enjoyed with it!

---

# Automate Let's Encrypt SSL on AWS Application Load Balancer

URL: https://ercan.cloud/automate-lets-encrypt-ssl-on-aws-application-load-balancer/
Date: 2022-07-02T18:09:07+03:00

In this article, you can automate all of Let's Encrypt issuing operations for AWS Application Load Balancer and learn how you can manage the..


Most of the time, we don't need anything except AWS Certificate Manager (ACM) but in some cases, if you are hosting your customer's domain with ALB and you don't have a chance to get an SSL certificate from your customer, you should do some workaround. In this article, we will issue a Let's Encrypt SSL on AWS Application Load Balancer and automatize it!

When you request an SSL certificate from Let's Encrypt, the LE has some verification methods such as web-challenge, route53, etc. In this scenario, I decided to use web-challenge because domains' DNS management is not on me. I just host the application and need to issue an SSL.

When you request an SSL with web-challenge, the Let's Encrypt is checking the https://domain.com/.well-known/acme-challenge path to make sure and verify. What does it mean? It means, that Let's Encrypt will send a request to the domain with port 80 and make verification. Hmm, we need do some adjusting on our Load Balancer config and need another Target Group for redirect the acme-challenge requests specifically.

Step 1: Run a new EC2 and Create a New Target Group

Clone The Repository

I wrote a bash script for easy management and keep the standards in the same way for every domain. You can check out my GitHub repository or clone the repository under the /opt/ path into acme-challange-server

Tip: # git clone https://github.com/flightlesstux/alble.git

Install nginx, Let's Encrypt and configure them

When we get a /.well-known/acme-challenge/* request, the nginx installed EC2 will handle all acme-challenge requests and issue an SSL for us.

Install nginx and configure for /.well-known/acme-challenge/

You can easily install nginx with amazon-linux-extras install nginx1 command. After the installation, you should put a new location block on the nginx roof config. The root config file is located in /etc/nginx/nginx.conf

location /.well-known/acme-challenge {
           root /opt/alble/certbot-challange;
        }

server_name value should be _; for all domains. We can't set a static DNS name.

Install Certbot with the nginx module

First of all, we will install certbot from epel repository and we need to install epel first. For epel installation, you can use amazon-linux-extras install -y epel command and after just run yum install -y nginx certbot python2-certbot-nginx jq dig command to cover pre-requisites.

IAM Role Policy for acme-challenge Server

This acme-challenge the server has an IAM Role for accessing ACM and ALB operations with Let's Encrypt SSLs. You can see the IAM Policy below.

{
    "Version": "2012-10-17",
    "Statement": &#91;
        {
            "Sid": "AWSLEPolicy1",
            "Effect": "Allow",
            "Action": &#91;
                "acm:DescribeCertificate",
                "acm:RemoveTagsFromCertificate",
                "acm:GetCertificate",
                "acm:AddTagsToCertificate",
                "acm:ListCertificates",
                "acm:ImportCertificate",
                "acm:ListTagsForCertificate"
            ],
            "Resource": "*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "acme-challenge-server-Elastic-IP/32"
                }
            }
        },
        {
            "Sid": "AWSLEPolicy2",
            "Effect": "Allow",
            "Action": &#91;
                "elasticloadbalancing:RemoveTags",
                "elasticloadbalancing:DescribeTags",
                "elasticloadbalancing:AddTags",
                "elasticloadbalancing:AddListenerCertificates"
            ],
            "Resource": "AWS::ALB::ARN",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "acme-challenge-server-Elastic-IP/32"
                }
            }
        },
        {
            "Sid": "AWSLEPolicy3",
            "Effect": "Allow",
            "Action": &#91;
                "elasticloadbalancing:DescribeSSLPolicies",
                "elasticloadbalancing:DescribeListeners",
                "elasticloadbalancing:DescribeListenerCertificates"
            ],
            "Resource": "*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "acme-challenge-server-Elastic-IP/32"
                }
            }
        },
        {
            "Sid": "AWSLEPolicy4",
            "Effect": "Allow",
            "Action": &#91;
                "elasticloadbalancing:RemoveTags",
                "elasticloadbalancing:DescribeTags",
                "elasticloadbalancing:AddTags"
            ],
            "Resource": "AWS::ALB::ARN",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "acme-challenge-server-Elastic-IP/32"
                }
            }
        },
        {
            "Sid": "AWSLEPolicy5",
            "Effect": "Allow",
            "Action": &#91;
                "elasticloadbalancing:DescribeSSLPolicies",
                "elasticloadbalancing:DescribeListeners",
                "elasticloadbalancing:AddListenerCertificates",
                "elasticloadbalancing:DescribeListenerCertificates"
            ],
            "Resource": "*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "acme-challenge-server-Elastic-IP/32"
                }
            }
        }
    ]
}

You can attach this policy directly to the acme-challenge server without any security issue and leak after assigning an Elastic IP to it. If you want to use local credentials, you can also use the same IAM Policy. 

Create a New Target Group

After EC2 instance configuration, I created a Target Group which name is acme-challenge and I redirect the request to the nginx installed EC2 which name is acme-challange-server and I register ours acme-challenge-server as a target. If everything goes without any problem, your screen will look like this below.

Step 2: Create an Application Load Balancer Listener Rule

We need to edit and create a new listener rule to accept and redirect the /.well-known/acme-challenge/* requests to the acme-challenge Target Group. When you did this step, you screen will look like this below.

Now, we are ready to request a Let's Encrypt SSL from ACME and issue without any problem and import to ACM and assign to AWS ALB : ) This operation contains three different steps for issuing.

Request an SSL from Let's Encrypt
Import requested SSL if domain or subdomain matched with AWS ALB CNAME record
Assign the imported SSL certificate to the AWS ALB for use in production

Let's Issue and Request a Let's Encrypt SSL from ACME

When you clone the git repository, you should move /opt/alble/renewal-hooks the folder to under /etc/letsencrypt/ path. Otherwise, you can't renew the certificate on the ACM and your and your customers' security be failed. You can copy and paste mv /opt/alble/renewal-hooks /etc/letsencrypt/

Just read the README.md file on the git repository and write your variables to the env file. After that, you can easily run/manage/create the Let's Encrypt SSL certificates on AWS Application Load Balancer. I recommend an AWS SSM to run the command to acme-challange-server for issue an SSL. For this, you should add AmazonEC2RoleforSSM the policy to your IAM Role.

I decided to use a subdomain for testing but it doesn't matter. You can also use a root domain. My test domains are awsle-1.ercanermis.com and awsle-2.ercanermis.com

When you are in /opt/alble/ path, you can run the command like ./create-new-site.sh awsle-1.ercanermis.com and you will see the output like below.

My ALBLe script is checking the CNAME record first for the domain and/or subdomain and making sure before continuing. If you write a wrong domain/subdomain or made a typo, it will warn you via Slack alerts. Alerts are really important to the automation things. You can set Slack Alerts on env file. Here is the example of if you issue the Let's Encrypt SSL before;

Production

My AWS ALB DNS name is web-application-elb-1302305711.us-east-1.elb.amazonaws.com and I can access it via https://web-app.ercanermis.com It's using AWS Certificate Manager to get an SSL.

My test domains are https://awsle-1.ercanermis.com and https://awsle-2.ercanermis.com and they are using Let's Encrypt with AWS Application Load Balancer, yeah! : )

https://web-app.ercanermis.com is using Amazon provided SSL
https://awsle-1.ercanermis.com is using Let's Encrypt provided SSL
https://awsle-2.ercanermis.com is using Let's Encrypt provided SSL
How AWS ALB SSL Certificates and ACM look like?

Here is the screenshot of ALB SSL Certificates
Here is the screenshot in Amazon Certificate Manager
I hope, this article will help you! P.S. Don't forget to set a cron for renewal requests.
Source code: https://github.com/flightlesstux/alble/

---

# Deploy a website to S3 and CloudFront with Bitbucket Pipelines

URL: https://ercan.cloud/deploy-a-website-to-s3-and-cloudfront-with-bitbucket-pipelines/
Date: 2022-06-23T11:44:00+03:00

In this article, you will learn how you can deploy your static website to the S3 Bucket and Cache Invalidation automatically...


Bitbucket is an Atlassian product and Git-based code store and CI/CD optimized tools for collaboration teams using with Jira. When you change your code and keep a revisions and history in Bitbucket, you also need to some automation for deployment. 

CI/CD is for every automation and it means Continuous Integration and Continues Delivery. In this article, we will use CI and CD in the same time.

I'm skipping "How to use Bitbucket?" and I'll introduce to you about Bitbucket pipelines for continues deployment to the AWS S3 which is using CloudFront for the distribution to all over the world. 

When you create static hosting supported S3 bucket, you should sync from your local folder to the S3 bucket. Normally, we should run the aws s3 sync LOCAL_FOLDER/ s3://STATIC_HOSTING_ENABLED_BUCKET_NAME --delete command for sync 100% local folder to the S3 Bucket.

If you are using a CloudFront for the distribution, you will see the old content when you visit the website. The reason is, CloudFront is caching the all content on all edge locations and serving the content from edge locations. We should "invalidate cache" for the new content and need to re-cache on the edge locations. 

Which IAM Policy for S3 and CloudFront to use?

For maximum security, I really love to put maximum restriction and minimum allowance for IAM Policies. You can see the policies below:

S3 Bucket IAM Policy

{
    "Version": "2012-10-17",
    "Statement": &#91;
        {
            "Sid": "S3BucketPolicy",
            "Effect": "Allow",
            "Action": &#91;
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": &#91;
                "arn:aws:s3:::BUCKET_NAME",
                "arn:aws:s3:::BUCKET_NAME/*"
            ],
            "Condition": {
                "ForAnyValue:IpAddress": {
                    "aws:SourceIp": &#91;
                        "34.199.54.113/32",
                        "34.232.25.90/32",
                        "34.232.119.183/32",
                        "34.236.25.177/32",
                        "35.171.175.212/32",
                        "52.54.90.98/32",
                        "52.202.195.162/32",
                        "52.203.14.55/32",
                        "52.204.96.37/32",
                        "34.218.156.209/32",
                        "34.218.168.212/32",
                        "52.41.219.63/32",
                        "35.155.178.254/32",
                        "35.160.177.10/32",
                        "34.216.18.129/32",
                        "3.216.235.48/32",
                        "34.231.96.243/32",
                        "44.199.3.254/32",
                        "174.129.205.191/32",
                        "44.199.127.226/32",
                        "44.199.45.64/32",
                        "3.221.151.112/32",
                        "52.205.184.192/32",
                        "52.72.137.240/32"
                    ]
                }
            }
        }
    ]
}

We will run our deployments in Bitbucket Pipelines and these IP addresses for Bitbucket Pipelines outbound IPs. You can see the reference from https://support.atlassian.com/bitbucket-cloud/docs/what-are-the-bitbucket-cloud-ip-addresses-i-should-use-to-configure-my-corporate-firewall/ 

CloudFront Cache Invalidation IAM Policy

For run the CloudFront cache invalidation command via Bitbucket Pipelines, you can use the IAM Policy below. You can attach the policy to the same AWS user which is allowed to S3 Sync operations.

{
    "Version": "2012-10-17",
    "Statement": &#91;
        {
            "Sid": "CloudFrontCacheInvalidationPolicy",
            "Effect": "Allow",
            "Action": "cloudfront:CreateInvalidation",
            "Resource": "arn:aws:cloudfront::ACCOUNT_ID:distribution/CF_DISTRIBUTION_ID",
            "Condition": {
                "ForAnyValue:IpAddress": {
                    "aws:SourceIp": &#91;
                        "34.199.54.113/32",
                        "34.232.25.90/32",
                        "34.232.119.183/32",
                        "34.236.25.177/32",
                        "35.171.175.212/32",
                        "52.54.90.98/32",
                        "52.202.195.162/32",
                        "52.203.14.55/32",
                        "52.204.96.37/32",
                        "34.218.156.209/32",
                        "34.218.168.212/32",
                        "52.41.219.63/32",
                        "35.155.178.254/32",
                        "35.160.177.10/32",
                        "34.216.18.129/32",
                        "3.216.235.48/32",
                        "34.231.96.243/32",
                        "44.199.3.254/32",
                        "174.129.205.191/32",
                        "44.199.127.226/32",
                        "44.199.45.64/32",
                        "3.221.151.112/32",
                        "52.205.184.192/32",
                        "52.72.137.240/32"

                    ]
                }
            }
        }
    ]
}

Bitbucket Pipelines

Previously, you already created a AWS user for run the Bitbucket Pipelines for CI/CD operations. The time is put the environment variables to the repository. The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY is enough.

Pipeline File

Now, in this step, we can create a bitbucket-pipelines.yml file for run the S3 Sync and CloudFront Cache Invalidation.

pipelines:
    branches:
      main:
        - step:
            name: Deploy to S3 Bucket
            deployment: Production
            script:
              - pipe: atlassian/aws-s3-deploy:1.1.0
                variables:
                  AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                  AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                  AWS_DEFAULT_REGION: AWS-REGION
                  S3_BUCKET: 'S3_BUCKET_NAME'
                  DELETE_FLAG: 'true'
                  LOCAL_PATH: './'
              - pipe: atlassian/aws-cloudfront-invalidate:0.1.1
                variables:
                  AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                  AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                  AWS_DEFAULT_REGION: AWS-REGION
                  DISTRIBUTION_ID: CLOUDFRONT-DIST-ID

 

When main branch has a new commit, pipeline will trigger and start to run immediately. After that, your Bitbucket Repository root path "/" will sync to your S3 Bucket and run the Cache invalidation immediately to apply new website to all edge locations.

I hope, this article will help you. If you want to ask anything, just leave a comment. I'll back to you as soon as possible I can.

---

# Protect your AWS Account with specified IPs

URL: https://ercan.cloud/protect-your-aws-account-with-specified-ips/
Date: 2022-06-16T13:18:15+03:00

You will learn, how you can protect your AWS account and the resources in the cloud with IAM Policy. Security is the first th...


Security is always important and the first thing before doing something in the cloud. I'm using the IP restriction policy for more than 5 years and I want to share this trick with you. There are two different ways to do it.

You should get a Static IP from your ISP and/or also you can also want to use your VPN IP address.

What is the IP Policy benefit?

The answer is simple. Nobody can access your resources in the cloud if they are not using the specified IPs.

Here is the example if the user logged in without a specific IP address in the IAM Policy. The user can able login but can not see anything about resources and/or information.

This is the example if the user logged in with a specific IP address. Yes, everything looks like okay and the user can see information about resources and other things.

Allow Access if your IPs Match

Here is the IAM IP Policy if you want to give permission to access to your resources.

{
    "Version": "2012-10-17",
    "Statement": &#91;
        {
            "Sid": "AllowAccessIfIPsMatch",
            "Effect": "Allow",
            "Action": "*",
            "Resource": "*",
            "Condition": {
                "ForAnyValue:IpAddress": {
                    "aws:SourceIp": &#91;
                        "1.2.3.4/32",
                        "5.6.7.8/32"
                    ]
                }
            }
        }
    ]
}

Deny Access if your IPs Don't Match

{
    "Version": "2012-10-17",
    "Statement": &#91;
        {
            "Sid": "DenyAccessIfIPsDontMatch",
            "Effect": "Deny",
            "Action": "*",
            "Resource": "*",
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": &#91;
                        "1.2.3.4/32",
                        "5.6.7.8/32"
                    ]
                }
            }
        }
    ]
}

This IAM policy also affected aws-cli requests and all AWS API calls. The Allow or Deny policy is the same (vice versa!) and you do not have to use both at the same time. I just want to inform you there is more than one way to protect your AWS account.

NOTE: This IAM Policy is just telling you about the IP restriction logic. Please think twice to use this IAM policy on your account and on the production because it allows giving an AdministratorAccess to all of AWS Resources!

Maybe you want to check out my previous blog post about "How to secure your Amazon Web Services account" before applying the IAM IP Policy. 

Thank you for reading!

---

# Differences between AWS CLI v1 and v2

URL: https://ercan.cloud/differences-between-aws-cli-v1-and-v2/
Date: 2022-06-09T23:35:00+03:00


Sometimes, aws-cli doesn't run your cli commands if you are using v1. If your aws cli is v1, I should recommend updating. In this article, you'll learn in which conditions you should use an AWS CLI v2.

AWS CLI v1 is written in Python language and for v1, you should install Python first but if you are using v2, it comes embedded and a Python interpreter is not needed

Auto Prompt feature is amazing! The AWS CLI v2 can prompt you for commands, parameters, and resources when you run an aws command.

You can import your config! Yes, AWS CLI v2 comes with aws configure import option and you can import your .csv credentials generated from the AWS Management Console. A .csv file is imported with the profile name matching the IAM user name. For example aws configure import.

Metadata Support for high-level S3 commands: You can configure additional metadata support for high-level S3 commands with the --copy-props argument.

AWS_REGION: is absolutely a lifesaver! It's an SDK-compatible variable and it overrides the AWS_DEFAULT_REGION variable on your environment. It's only available in the AWS CLI.

ddb put and ddb select commands are available on AWS CLI v2 for DynamoDB operations. You can put the items into DynamoDB and search in DynamoDB tables or indexes.

YAML Support: Previously, AWS CLI v1 is only using JSON but in v2, it's more dynamic for
stream-formats and the yaml.

aws configure list-profiles: You can list all aws cli profiles with one command.

Docker Support: This means you don't have to install AWS CLI v2 anymore. You can just run the docker container and run your aws cli commands easily. You can run into your CI/CD directly without no worries.

aws logs tail: For CloudWatch Logs groups, you can use aws logs tail command to tails the logs. By default, the command returns logs from all CloudWatch Logs streams during the last 10 minutes.

Single Sign-On: If you are an enterprise user, you probably use an SSO to login to AWS Console. Now, AWS CLI v2 is supporting IdP for sign in requests. For example, maybe you are using the Active Directory or Google Cloud SSO Service. You can also use it via AWS CLI v2.

Client-Side Pager Program: The AWS CLI v2 provides the use of a client-side pager program for output. Default, this feature comes with turned on and returns all output through into your OS default pager program.

Wizards: This is really interesting feature I guess. You can use wizard for some configure, dynamodb, events, iam and lambda. You can follow the example on GitHub. https://github.com/aws/aws-cli/tree/v2/awscli/customizations/wizard/wizards 

I hope this article helped you.

---

# SSL CA Problem on CentOS7 Docker and Solution

URL: https://ercan.cloud/ssl-ca-problem-on-centos7-docker-and-solution/
Date: 2022-03-08T08:19:26+03:00


When I'm playing with CentOS7 docker image on MacBookPro M1, it looks like it doesn't connect to the servers which require HTTPS connection. 

For example, when I run yum -y update the command it returns curl: (77) Problem with the SSL CA cert (path? access rights?). Hmm, it looks like an OS Bug because my ca-certificates are okay.

I tried to send a GET command via curl and it looks like doesn't work.

$ curl -vvv https://google.com
* About to connect () to google.com port 443  ( # 0) 
* Trying 216.58.212.142 ...
* Connected to github.com (216.58.212.142) port 443 (#0)
* Initializing NSS with certpath: sql: / etc / pki / nssdb
* Closing connection 0 curl: ( 77 ) Problem with the SSL CA cert ( path? Access rights? )

wget also doesn't work with HTTPS connections and I do some research online and I see it's a real bug. Here is the details; https://bugs.centos.org/view.php?id=16282

The Solution

$ touch "/etc/sysconfig/64bit_strstr_via_64bit_strstr_sse2_unaligned"

Yes, that's it. We just need this file to run TCP connections securely. Here is the results after the solution.

$ curl -I https://google.com
HTTP/2 301
location: https://www.google.com/
content-type: text/html; charset=UTF-8
date: Tue, 08 Mar 2022 08:16:51 GMT
expires: Tue, 08 Mar 2022 08:16:51 GMT
cache-control: private, max-age=2592000
server: gws
content-length: 220
x-xss-protection: 0
x-frame-options: SAMEORIGIN
set-cookie: CONSENT=PENDING+370; expires=Thu, 07-Mar-2024 08:16:51 GMT; path=/; domain=.google.com; Secure
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

It is also a mystery that it is reproduced only on the M1 machine so far. But for the time being, I was able to identify the cause, and it's okay to reach the goal, right?

---

# What do I have?

URL: https://ercan.cloud/what-do-i-have/
Date: 2021-07-20T15:02:36+03:00

Apple MacBookPro M1, Dell XPS13 4K, Juniper SRX-110, Xiaomi 4A Router, Dell Pivot Monitor, Logitech MX Keys, Logitech MX Master 3.


First of all, hello from sunny Izmir. Today is Tuesday but I feel like off-day. I know, I need to write technical articles for the community that is working with passionate people. It's on the way, I promise... The title is "Cloud: Using NAT on Private Subnets". The article is comparing serverless NAT service and self-managed ec2 NAT service. Anyway, let's start...

For Work: Apple MacBookPro M1 with 16GiG memory and a 256GB SSD. OS: Always latest. For now macOS Big Sur Version 11.4

For Personal: Dell XPS 13 with 4K Touch Screen resolution. It has 8 GiG memory and a 256 GB NVMe disk on it. OS Info: Manjaro Linux with KDE up-to-date.

For Other Purposes: Intel® NUC Kit NUC8i7HVK. It's really amazing hardware on it. You can see details on the product page. It has 32 GiG memory with 2 x 512 GB RAID 0 NVMe disk. Windows 10 Enterprise and I are really fell in love with it.

Since 2017, I'm using Samsung UE28E590, 4K 4ms monitor. It's not a pivot and I used Dell P2719H 27" 5ms FHD as a pivot for my SSH sessions but that monitor owner is my previous company and I give back when I decided to leave.

My company gives me an Apple Magic Mouse 2 and Apple Wireless Keyboard and now I'm using them and I'm okay with that.

My Internet provider is turk.net and turk.net is a really amazing provider with cost-effective plans. I'm using VDSL2 internet technology at home (WFH) and I'm getting 100mbps download and 8mbps upload speed. I can configure this monster to connect via IPSec VPN to AWS Cloud and/or Google Cloud Platform. I'm also testing some services and using them if I need. Device is Juniper SRX-110HA v2. Oldies but goldies...
I'm using Xiaomi 4A Router with 5GHz mode. Just my TV directly cable connected to my local network. Other mobile devices connecting via Wi-Fi.
I'm using Samsung UE50RU7400UXTK (2019). It's 50" and it's fair enough for me. Mostly I'm watching YouTube and listening on Spotify. The platform is Tizen and OS is also running on the latest version. The AirPlay2 feature is really a life saver for me. I'm connecting on MBP with few clicks.
Now I'm planning to use pivot monitor again for my CLIOps and SSH sessions.
It's Dell P2418D with 2K resolution.
Next week, I'm ordering these amazing kit and I'm switching Apple Magic Mouse 2 and Apple Keyboard set.

---

# I'm officially AWS Community Builder!

URL: https://ercan.cloud/im-officially-aws-community-builder/
Date: 2021-05-15T09:20:41+03:00

This morning, I've got an e-mail from @JasonDunn about the AWS Community Builder program. I don't remember when I applied to this program but


Every day I'm using AWS because I'm really open to educating myself and AWS helps me in this way. I'm automating the routine tasks for departments like client team, back-end team, sometimes data team, etc., and most of the time, needs are specific for that organization. So, you should build on your own and/or with your team and improve your product and/or tool knowledge when you are on the way. 

If you share your knowledge, you can save people day and time. Most of the time, people are stuck in the same thing. If you solve the problem, why won't you share the solution? You don't need this anymore because you already know after all. This is my main motivation for the open-source community and AWS Community. 

AWS Community is also open-source. For example, the AWS Command Line Interface Documentation page is hosting on GitHub and you can contribute if something wrong. This means it's open source.

And also, I really like to write Terraform for cloud automation, if I see a problem on any module, provider, or unexpected behavior from the tool, I'm checking the Issue tab on GitHub for that thing. For example, https://github.com/hashicorp/terraform-provider-aws/issues. The feeling is amazing. 

Today, if I'm standing here, the main reason is the open-source community. Every time I have a chance to try to run open-source projects on Linux Servers, creating clusters, trying to understand HA scenarios, resilient systems, immutable infrastructures, etc. There was an on-demand and if you did something wrong at the kernel side, most of the time the server won't boot up after a reboot! : ) And after that, you are spending too many hours but getting knowledge at the end of the story. In the cloud era, you can create an autoscaling infrastructure (ec2, docker, serverless, etc.) with a load balancer and secure like a castle with AWS easily and super fast! That kind of infrastructure is a few-click long from your fingers...

If you are using a tool, you should know how you can use that tool. You should get knowledge before you start but on the cloud side, you don't have to know before you start. You can create your own playground are in AWS and start the learning.

This morning, I've got an e-mail from @JasonDunn about the AWS Community Builder program. I don't remember when I applied to this program but I checked my e-mails and I found a conversation with Jason around March 2021. It should be March because the application was open until midnight Pacific time (GMT -7) on March 31st, 2021. I've really appreciated it! 

Let's make a state-of-art together!

---

# Deploy HA nginx to AWS ECS with Geolocation Routing via Terraform

URL: https://ercan.cloud/deploy-ha-nginx-to-aws-ecs-with-geolocation-routing-via-terraform/
Date: 2021-05-12T12:56:20+03:00

You can deploy the nginx docker image in two different regions in AWS. If you need, you can add more region support by yout self...


Today, I want to refactor the Deploy nginx docker to AWS ECS with Terraform Automation project from a single region to multi-region support. This time, I pushed all of Terraform source code to my GitLab account. 

Now, the updated project is creating all infrastructure in Ireland and N.Virginia region.
Now, we have a geo-location-based route policy which is provided via Route53.

What geo-location-based route policy means?

In this project, When you get a request from the North America continent, the visitors' request will be handle by North Virginia's infrastructure.

If you get a request from Europe continent, this time the request will be handle by Ireland's infrastructure.

The default region is also set and handle by Ireland's infrastructure.

Bonus

This project is really funny for me. I've spent $1.17 (with tax) on AWS for the PoC. Here is the repository link: https://gitlab.com/flightlesstux/nginx-app-aws-ecs 

Don't forget to check my pipelines via https://gitlab.com/flightlesstux/nginx-app-aws-ecs/-/pipelines/301488894 !

---

# Deploy nginx docker to AWS ECS with Terraform Automation

URL: https://ercan.cloud/deploy-nginx-docker-to-aws-ecs-with-terraform-automation/
Date: 2021-05-11T07:40:23+03:00

You can understand how you can deploy nginx docker to AWS ECS with terraform easily. The terraform will handle everything what you need.


Last night, I want to play with AWS ECS Service. I wrote a terraform code to deploy everything at the same time. The terraform is creating all of the resources for run ECS in different VPC and isolated from your other resources.

I used nginx:alpine docker image for this project but if you need, you can modify or change the terraform and implement it for your needs.

What is AWS ECS?

Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service. Customers use ECS to run their most sensitive and mission-critical applications because of its security, reliability, and scalability.

ECS is a great choice to run containers for several reasons. First, you can choose to run your ECS clusters using AWS Fargate, which is serverless compute for containers. Fargate removes the need to provision and manage servers, lets you specify and pay for resources per application, and improves security through application isolation by design. Second, ECS is used extensively within Amazon to power services such as Amazon SageMaker, AWS Batch, Amazon Lex, and Amazon.com’s recommendation engine, ensuring ECS is tested extensively for security, reliability, and availability.

Additionally, because ECS has been a foundational pillar for key Amazon services, it can natively integrate with other services such as Amazon Route 53, Secrets Manager, AWS Identity and Access Management (IAM), and Amazon CloudWatch providing you a familiar experience to deploy and scale your containers. ECS is also able to quickly integrate with other AWS services to bring new capabilities to ECS. For example, ECS allows your applications the flexibility to use a mix of Amazon EC2 and AWS Fargate with Spot and On-Demand pricing options. ECS also integrates with AWS App Mesh, which is a service mesh, to bring rich observability, traffic controls, and security features to your applications. ECS has grown rapidly since launch and is currently launching 5X more containers every hour than EC2 launches instances.

Repository

You can check the repository via https://github.com/flightlesstux/aws-ecs-nginx-app and If you need, you can use the code to make deploy or understand how ECS is running.

---

# How to Install Node Exporter on Linux Server

URL: https://ercan.cloud/how-to-install-node-exporter-on-linux-server/
Date: 2021-05-10T09:18:09+03:00

You can monitor your Linux Server easily with Node Exporter. Node Exporter installation it little bit different but just follow the article..


Node Exporter is a Prometheus exporter for server level and OS level metrics with configurable metric collectors. It helps us in measuring various server resources such as RAM, disk space, and CPU utilization. Node exporter is a good solution to collect all the Linux server-related metrics and statistics for monitoring.

Before Start

Prometheus Server
After node exporter installation, you should send your metrics to the Prometheus server.
Sudo Privileges
You should access your server via SSH with root or your ssh login user should have root privileges with sudo.

Node Exporter Installation Steps

 Create a directory I prefer to use /opt/node-exporter path for installation. You can also use.

 
Add user You can add a user with a simple command. It's:
adduser -r -d /opt/node-exporter node-exporter -s /sbin/nologin

 
Download the binary Download the binary from https://github.com/prometheus/node_exporter/releases page and extract the files to /opt/node-exporter path directly.

 
Change Owner and Group Permissions Now, need to set user permissions to execute node_exporter binary. 
chown -R node-exporter:node-exporter /opt/node-exporter

 
Create a systemd service file We should manage this service status with systemd. Now, time to create a systemd service file. The file path should /etc/systemd/system/node-exporter.service You can paste all configurations below into it.

[Unit]
Description=Prometheus exporter for machine metrics
Documentation=https://github.com/flightlesstux/node_exporter

[Service]
Restart=always
User=node-exporter
Gorup=node-exporter
ExecStart=/opt/node-exporter/node_exporter
ExecReload=/bin/kill -HUP $MAINPID
TimeoutStopSec=20s
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

 
Start the node-exporter Now, you can ready to start your node-exporter service with systemctl start node-exporter command.

 

Now, you can see your servers metrics via http://IP_ADDRESS:9100/metrics url. If you can't reach the page, you should check your firewall and/or security group settings to allow TCP 9100.

Bonus

Now, you now the steps and what you need for install Node Exporter. I turned this installation to automation and you can feel free to use. Just run the command sh <(curl https://raw.githubusercontent.com/flightlesstux/node_exporter/master/installer.sh) and in a few seconds, your node-exporter will be installed.

GitHub Repository: https://github.com/flightlesstux/node_exporter

---

# FortiClient Problem on M1 MacBookPro Problem Solution

URL: https://ercan.cloud/forticlient-problem-on-m1-macbookpro-problem-solution/
Date: 2021-05-05T06:36:38+03:00

I found a quick fix solution for FortiClient VPN client connectivity issue with Apple M1 MacBookPro. It's not a big deal...


I need to connect VPN via FortiClient to do my work and access some resources securely. When I started to work for my company in March 2021, they gave me an M1 MacBookPro. M1 MacBookPro is different than the others because it comes with a new CPU architecture. Previously, Apple was using an Intel-based CPU but now, they decided to run macOS with an ARM-based CPU and Apple called M1 for now.

A lot of software not compatible with ARM. Sometimes I'm having some issues like "This component is not compatible with your CPU".

When I connected to VPN via FortiClient v6.4.3.1325, it looks like connected but my internet speed is getting too slow. Normally, I have 100MB/sec internet speed. I asked my colleagues about "are you having any issues with VPN and/or VPN speed?" and that they said "NO!" They are using a little bit older MacBookPro than mine with an Intel-based CPU and I thought it's a normal situation because Intel-based versions are okay and stable.

I started to debugging to find a root cause and I checked the routing table before connect to VPN.

➜  ~ netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            192.168.31.1       UGScg          en0
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0
169.254            link#14            UCS            en0      !
192.168.31         link#14            UCS            en0      !
192.168.31.1/32    link#14            UCS            en0      !
192.168.31.1       ec:41:18:ec:c6:bc  UHLWIir        en0   1189
192.168.31.147/32  link#14            UCS            en0      !
192.168.31.147     a0:78:17:87:b4:88  UHLWI          lo0
192.168.31.171     b8:bc:5b:6:28:18   UHLWI          en0   1165
192.168.31.255     ff:ff:ff:ff:ff:ff  UHLWbI         en0      !
224.0.0/4          link#14            UmCS           en0      !
224.0.0.251        1:0:5e:0:0:fb      UHmLWI         en0
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI         en0
255.255.255.255/32 link#14            UCS            en0      !

My route table is looking okay. 192.168.31.1 is my wireless router and default root it to my wireless router. It's okay.

After this, I connected to VPN via FortiClient and I re-checked my route table. 10.212.134.152 is my local IP address which one is assigned by FortiClient.

➜  ~ netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            link#19            UCSg          ppp0
default            192.168.31.1       UGScIg         en0
8.8.8.8            link#19            UHWIig        ppp0
13.224.58.179      link#19            UHWIig        ppp0
31.XXX.XX.XX/32    192.168.31.1       UGSc           en0
34.XXX.XX.XXX      link#19            UHWIig        ppp0
34.XXX.XXX.xxx     link#19            UHWIig        ppp0
35.XXX.XXX.XX      link#19            UHWIig        ppp0
35.XXX.XXX.XXX     link#19            UHWIig        ppp0
80.80.80.80        link#19            UHW3Ig        ppp0   3597
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0
169.254            link#14            UCS            en0      !
169.254.38.179     10.212.134.152     UH            ppp0
192.168.31         link#14            UCS            en0      !
192.168.31.1/32    link#14            UCS            en0      !
192.168.31.1       ec:41:18:ec:c6:bc  UHLWIir        en0   1181
192.168.31.147/32  link#14            UCS            en0      !
192.168.31.147     a0:78:17:87:b4:88  UHLWI          lo0
192.168.31.171     b8:bc:5b:6:28:18   UHLWIi         en0   1139
192.168.31.255     ff:ff:ff:ff:ff:ff  UHLWbI         en0      !
224.0.0/4          link#19            UmCS          ppp0
224.0.0/4          link#14            UmCSI          en0      !
224.0.0.251        link#19            UHmW3I        ppp0   3599
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI         en0
239.255.255.250    link#19            UHmW3I        ppp0   3599
255.255.255.255/32 link#19            UCS           ppp0
255.255.255.255/32 link#14            UCSI           en0      !

Our VPN service is not using split-tunneling and link#19 OR 10.212.134.152 is should be the next hop for the default route with my wireless router at the same time. 

I delete my default route!

Yes, I decided to manipulate my routing table manually. 

➜  ~ sudo route delete default
Password:
delete net default

Now, the next hop should be my FortiClient local ip address. It's 10.212.134.152 for my case. This local ip address is dynamic and it changes on every connection because DHCP is assigning this ip address.

Now, how my route table looks like? The default route shouldn't be there...

➜  ~ netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            192.168.31.1       UGScIg         en0
31.XXX.XX.XX/32    192.168.31.1       UGSc           en0
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0
169.254            link#14            UCS            en0      !
169.254.38.179     10.212.134.152     UH            ppp0
192.168.31         link#14            UCS            en0      !
192.168.31.1/32    link#14            UCS            en0      !
192.168.31.1       ec:41:18:ec:c6:bc  UHLWIir        en0   1178
192.168.31.147/32  link#14            UCS            en0      !
192.168.31.147     a0:78:17:87:b4:88  UHLWI          lo0
192.168.31.171     b8:bc:5b:6:28:18   UHLWIi         en0    586
192.168.31.222     c:2c:54:e4:67:81   UHLWI          en0   1156
192.168.31.255     ff:ff:ff:ff:ff:ff  UHLWbI         en0      !
224.0.0/4          link#19            UmCS          ppp0
224.0.0/4          link#14            UmCSI          en0      !
224.0.0.251        link#19            UHmW3I        ppp0   3589
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI         en0
239.255.255.250    link#19            UHmW3I        ppp0   3588
255.255.255.255/32 link#19            UCS           ppp0
255.255.255.255/32 link#14            UCSI           en0      !

link#19 is disappeared from default route. Nice! And now, I can't access to the internet. As a default route, my wireless router is still on the route table but FortiClient is using it to keep connected to the VPN. Yeah, I'm still connected to the internet but I don't have a next hop.

➜  ~ ping -c 4 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: sendto: No route to host
ping: sendto: No route to host
Request timeout for icmp_seq 0
ping: sendto: No route to host
Request timeout for icmp_seq 1
ping: sendto: No route to host
Request timeout for icmp_seq 2

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

Let's add a new default route...

➜  ~ sudo route add default 10.212.134.152
add net default: gateway 10.212.134.152

Now, I can able to ping 8.8.8.8 after add a new default route.

➜  ~ ping -c 4 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=114 time=99.007 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=98.277 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=114 time=103.634 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=114 time=104.365 ms

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 98.277/101.321/104.365/2.704 ms

Cross check

I've checked the route table for last time to see what is difference after delete and add default route.

➜  ~ netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            10.212.134.152     UGScg         ppp0
default            192.168.31.1       UGScIg         en0
31.145.77.18/32    192.168.31.1       UGSc           en0
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0
169.254            link#14            UCS            en0      !
169.254.38.179     10.212.134.152     UH            ppp0
192.168.31         link#14            UCS            en0      !
192.168.31.1/32    link#14            UCS            en0      !
192.168.31.1       ec:41:18:ec:c6:bc  UHLWIir        en0   1179
192.168.31.147/32  link#14            UCS            en0      !
192.168.31.147     a0:78:17:87:b4:88  UHLWI          lo0
192.168.31.171     b8:bc:5b:6:28:18   UHLWI          en0   1146
192.168.31.222     c:2c:54:e4:67:81   UHLWI          en0   1163
192.168.31.255     ff:ff:ff:ff:ff:ff  UHLWbI         en0      !
224.0.0/4          link#19            UmCS          ppp0
224.0.0/4          link#14            UmCSI          en0      !
224.0.0.251        link#19            UHmW3I        ppp0   3582
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI         en0
239.255.255.250    link#19            UHmW3I        ppp0   3557
255.255.255.255/32 link#19            UCS           ppp0
255.255.255.255/32 link#14            UCSI           en0      !

Now, everything is running perfect. My VPN connection is stable and fast!

Bonus

I need to delete and add a new default route when I need to connect VPN and every day I need to connect. I don't want to delete and add a new route manually and I wrote a little bash script for this routine.

You can save the script into your /usr/local/bin/ path with sudo vim. After that, don't forget to add chmod permissions for the script.

#!/bin/bash

FortiIP=$(netstat -rn | grep "10.212.134" | awk '{print $2}')

sudo route delete default
sudo route add default $FortiIP

If you read from begging, I should say thank you! : )

---

# Connect your AWS to GCP with Terraform via IPSec Site-to-Site VPN

URL: https://ercan.cloud/connect-your-aws-to-gcp-with-terraform-via-ipsec-site-to-site-vpn/
Date: 2021-05-04T20:49:25+03:00

You can deploy the VPN infrastructure easily with this article. Terraform is a automation tool you can do everyting in the cloud era.


Today, I wrote another IaaC (infrastructure as a code) for my needs. You can create a secure VPN connection (IPsec) between Amazon Web Services (AWS) and Google Cloud Platform (GCP).

What can this do?

Creating a Customer Gateway on AWS. Creating a Virtual Private Gateway on AWS. Creating a Site-to-Site VPN Connection on AWS. Creating a Security Group for VPN connection access on AWS. Creating an External IP address for VPN connection on GCP. Creating a Managing the firewall rule for VPN connection on GCP. Creating a Route rule on GCP.

Requirements

Enable Compute Engine API if doesn't enable.
Create a Service account with Editor role (or whatever you want) and export the key file.
Create Cloud Google Storage Bucket to keep the terraform state. If you want, you can add your service account as a member to the bucket OR Create S3 Bucket to keep the terraform state.
Check your IAM Permissions on the AWS side.

Usage

 Clone the repositry git clone git@github.com:flightlesstux/aws-to-gcp-vpn-w-terraform.git

 
Authenticate to Google Platform if even google-sdk is not installed. It's really useful for CI/CD pipelines! If you already logged in to your Google Cloud Platform project like gcloud auth login, you can skip this step. export GOOGLE_APPLICATION_CREDENTIALS="service-account-key.json"

 
Project ID is also declared via terraform but if you don't export the value, you will probable get an error like below. export GOOGLE_PROJECT="0123456789012"

│ Error: project: required field is not set
│
│ with google_compute_instance_group_manager.this,
│ on group_manager.tf line 1, in resource "google_compute_instance_group_manager" "this":
│ 1: resource "google_compute_instance_group_manager" "this" {

 
Export your AWS Access Key ID and Secret Access Key to create a resource in your AWS Region. AWS_ACCESS_KEY_ID=AKIA1SFAESADASFASR5D
AWS_SECRET_ACCESS_KEY=Aasdfiajfar1O9DFASDAA3rasdas02304adsq9re

 
Set your variables Edit values for variables.tf

 
State bucket settings Set your terraform state bucket via state.tf file. You can use AWS S3 or Google Cloud Storage.

 
Initialize terraform init

 
Deploy! terraform apply or terraform apply -auto-approve

 

Repository Address

https://github.com/flightlesstux/aws-to-gcp-vpn-w-terraform

I hope, you enjoyed with this article...

---

# Google Cloud Platform Automation with Terraform Easily

URL: https://ercan.cloud/google-cloud-platform-automation-with-terraform-easily/
Date: 2021-04-30T08:51:51+03:00

You can deploy your load balancer, instance template, instance group and firewall rules easily with terraform.


Once upon a time, when I was learning Google Cloud Platform, I just want to automate my workflow and deployments with terraform and, I wrote this IaaC (Infrastructure as a Code) to manage my infrastructure easily and fastly.

What can this do?

Creating a secure load balancer with SSL and turns the HTTP requests to HTTPS with 301 redirection code.
Creating an Instance Template and set everything about network, subnet, startup script.
Creating an Instance Group and manage them. If your instance group CPU is more or less than a value, the instance group manager will automatically scale up or scale down the VMs.
VM instances are running with private IP and, project-wide SSH access is not allowed on default. Managing the firewall rule for load balancer health check.

Requirements

Enable Compute Engine API if doesn't enable.
Create Service account with `Editor role (or whatever you want) and export the key file.
Create Cloud Google Storage Bucket for keep the terraform state. If you want, you can add your service account as a member to bucket.
Create Compute Engine Image You can create a Debian 10 based VM and install nginx via sudo apt install -y nginx and test this terraform code. Don't forget to sudo systemctl enable nginx command before turn your VM to image.

Usage

Here is the steps...

 Clone the repository  
git clone git@github.com:flightlesstux/gcp-instance-group-and-loadbalancer-w-terraform.git

 
Authenticate to Google Platform if even google-sdk is not installed. It's really useful for CI/CD pipelines! If you already logged in to your Google Cloud Platform project like gcloud auth login, you can skip this step. 
export GOOGLE_APPLICATION_CREDENTIALS=service-account-key.json

 
Project ID is also declared via terraform but if you don't export the value, you will probable get an error like below. 
export GOOGLE_PROJECT="0123456789012"

│ Error: project: required field is not set
│
│ with google_compute_instance_group_manager.this,
│ on group_manager.tf line 1, in resource "google_compute_instance_group_manager" "this":
│ 1: resource "google_compute_instance_group_manager" "this" {

 

 
Edit variable values  
variables.tf file.

 
SSL Certificate Settings 
Paste your SSL certificate and the key file under certs folder . Loadbalancer needs to be secure! You can paste your Public key certificate and Certificate chain into certificate.crt.

 

 
You can follow-up your production environment version easily with this variable. And also, Instance group need the new version for make a rolling update.  
export TF_VAR_deploy_version=v1

 
Initilaze 
terraform init

 
Let's Deploy 
terraform apply or terraform apply -auto-approve

 

Repository Address

https://github.com/flightlesstux/gcp-instance-group-and-loadbalancer-w-terraform

I hope this article will help you.

---

# How to secure your Amazon Web Services account

URL: https://ercan.cloud/how-to-secure-your-amazon-web-services-account/
Date: 2021-04-24T14:27:03+03:00

Learn how to secure your Amazon Web Services account. Root level account has got full access to all resources running in the cloud enviro...


Security first because security is so important! When you create an Amazon Web Services account, that account called a root. Root level account has got full access to all resources running in the cloud environment and I totally recommended don't use your root account to manage the resources.

What should you do to keep your AWS Account secure?

 Set Multi Factor Authentication for root account Maybe you heard 2 Step Authentication or 2 Factor Authentication previously. In the Amazon ecosystem, we called this extra security layer an MFA. The root user for this account does not have Multi-factor authentication (MFA) enabled. Enable MFA to improve security for this account.

 
Set password policy A password policy is a set of rules that define complexity requirements and mandatory rotation periods for your IAM users' passwords.

You can set:
- Enforce minimum password length
- Require at least one uppercase letter from Latin alphabet (A-Z)
- Require at least one lowercase letter from Latin alphabet (a-z)
- Require at least one number
- Require at least one non-alphanumeric character (! @ # $ % ^ & * ( ) _ + - = [ ] { } | ')
- Enable password expiration
- Password expiration requires administrator reset
- Allow users to change their own password
- Prevent password reuse

 
Deactive Region(s) if you don't need In Account Settings, every single region running specifically in Amazon Web Services. For example, if you are planning to run your services in the London region (eu-west-2), actually you don't need other regions and you should definitely deactivate others. You can't deactivate Global Endpoint and US East (N. Virginia) regions. The all Amazon Web Services authentication services (such as logins and other service authentications) running in Global Endpoint, some of the services just running in the North Virginia region (us-east-1). That's the reason you cannot disable them.

 
Create Policy A policy defines the AWS permissions that you can assign to a user, group, or role. You can create and edit a policy in the visual editor and/or using JSON.

You can set specific actions for specific services and resources on create policy page.

 
Create IAM Role IAM roles are a secure way to grant permissions to entities that you trust. Examples of entities include the following:
- IAM user in another account
- Application code running on an EC2 instance that needs to perform actions on AWS resources
- An AWS service that needs to act on resources in your account to provide its features
- Users from a corporate directory who use identity federation with SAML
- IAM roles issue keys that are valid for short durations, making them a more secure way to grant access.

 
Separate your departments with User Groups  An IAM user is an entity that you create in AWS to represent the person or application that uses it to interact with AWS. A user can belong to up to 10 groups. You can attach up to 10 policies to this user group. All the users in this group will have permissions that are defined in the selected policies.

 
Apply the best practices 
Grant least privilege access: Establishing a principle of least privilege ensures that identities are only permitted to perform the most minimal set of functions necessary to fulfill a specific task while balancing usability and efficiency.

Use AWS Organizations: Centrally manage and govern your environment as you scale your AWS resources. Easily create new AWS accounts, group accounts to organize your workflows, and apply policies to accounts or groups for governance.

Enable Identity federation: Manage users and access across multiple services from your preferred identity source. Using AWS Single Sign-On centrally manages access to multiple AWS accounts and provides users with single sign-on access to all their assigned accounts from one place.

Enable MFA: For extra security, we recommend that you require multi-factor authentication (MFA) for all users.

Rotate credentials regularly: Change your own passwords and access keys regularly, and make sure that all users in your account do as well.

Enable IAM Access Analyzer: Enable IAM Access Analyzer to analyze public, cross-account, and cross-organization access.

Learn more about all security best practices.

 

I hope, this article will help you to use your Amazon Web Service account securely!

---

# Install UGREEN USB Ethernet Adapter on macOS

URL: https://ercan.cloud/install-ugreen-usb-ethernet-adapter-on-macos/
Date: 2021-04-18T14:39:34+03:00

Today, I need to connect my Apple MacBookPro Silicon to the internet via the same USB adapter but macOS doesn't like this.


In 2019, I need to connect my Dell XPS13 to my Juniper SRX-110 via ethernet cable but on my Dell, it doesn't have any ethernet port on it. I decided to buy UGREEN USB Ethernet Adapter from Amazon.

The ethernet adapter is driver-free and it works on Windows and Linux perfectly. Today, I need to connect my Apple MacBookPro Silicon to the internet via the same USB adapter but macOS doesn't like this. I know, MacOS needs a driver and some kernel limitations for some additional hardware. 

Kext files are essential drivers for macOS "Kext" stands for Kernel Extension; kext files "extend" Mac OS X's kernel, the core part of the operating system, by providing additional code to be loaded when your computer boots.

I need to disable System Integrity settings without kext. I booted my Mac on recovery mode and I jumped to Terminal and run the command below.

$ csrutil enable --without kext

After csrutil command, I typed my username and password which one has Administrator access and I rebooted my Apple Silicon.

Now time is to install the chipset driver to macOS Big Sur. I checked the chipset name on System Report and I see the chipset name is AX88179.

After few search, I found the chipset support page and I downloaded driver from https://www.asix.com.tw/en/product/USBEthernet/Super-Speed_USB_Ethernet/AX88179 

Now, my MacBookPro is also able to connect to the internet via my UGREEN USB Ethernet Adapter directly!

You can check UGREEN branded USB Ethernet Products. I hope, this article will help you too...

---

# Redirect 301 HTTPS on App Engine with nginx on Google Cloud Platform

URL: https://ercan.cloud/redirect-301-https-on-app-engine-with-nginx-on-google-cloud-platform/
Date: 2021-04-15T07:06:42+03:00

You can learn how to redirect http requests turn to https easily. AppEngine and nginx will work smoothly after this tutorial.


I started to play with App Engine on Google Cloud Platform these days. I'm using a flexible environment with a custom runtime to run PHP in Docker. App Engine serving the content via HTTP and HTTPS as well. I checked the disable HTTP or redirect the HTTP traffic easily to HTTPS via Web UI but unfortunately, it's not possible right now. Yes, you heard right, the basic feature is not supporting!

I check the document for (https://cloud.google.com/appengine/docs/standard/php7/application-security) I tried to set a 301 redirect via app.yaml file. Here is what I tried:

handlers:
- url: /.*
  script: auto
  secure: always
  redirect_http_response_code: 301

I deployed a new version of my app but App Engine still serving via HTTP and it doesn't redirect to HTTPS when I requested insecure connection.

This time, I decided to give a shot to nginx. I opened my nginx-app.conf file and I wrote this:

server_name _;

if ($http_x_forwarded_proto = "http") {
    return 301 https://$host$request_uri;
}

And Voila! It worked so perfectly. How I tried and understand this process you can see below.

➜  ~ curl -I http://ercan-ermis.ew.r.appspot.com/
HTTP/1.1 301 Moved Permanently
Date: Thu, 15 Apr 2021 07:03:34 GMT
Content-Type: text/html
Content-Length: 178
Server: nginx
Location: https://ercan-ermis.ew.r.appspot.com/
Via: 1.1 google

---

# Set two different Target Groups on AWS Load Balancer with Terraform

URL: https://ercan.cloud/set-two-different-target-groups-on-aws-load-balancer-with-terraform/
Date: 2021-04-05T11:18:19+03:00

You will learn how you can use more than one target group behind Elastic Load Balancer in aws services with Terraform.


Last week, I wrote an infrastructure as a code via Terraform. In my case, I need to forward the traffic from AWS Application Load Balancer public interface to two different target groups. The hashicorp language (HCL) not a hard language but the document is missing a lot of subjects and also there are some points in terraform are undocumented.

I've Google it before the make sure but I've seen there is a lot of questions and issues about the "Use more than one target group on AWS with terraform" subject.

I read the suggested solutions but they do not help me. After that, I decided to use "aws_lb_listener" as a resource. Finally, I found a solution to my case for my own. You can see details below:

resource "aws_lb" "awsalb" {
  name                       = "my-aws-loadbalancer"
  internal                   = false
  load_balancer_type         = "application"
  security_groups            = &#91;aws_security_group.lb.id]
  subnets                    = data.aws_subnet_ids.subnets.ids
  enable_deletion_protection = false

  tags = {
    Name     = "My AWS ALB Load Balancer"
  }
}

resource "aws_lb_target_group" "origin" {
  name     = "Origin-Group"
  port     = 5080
  protocol = "HTTP"
  vpc_id   = data.aws_vpc.default.id

  stickiness {
      type = "lb_cookie"
  }
}

resource "aws_lb_target_group" "edge" {
  name     = "Edge-Group"
  port     = 5080
  protocol = "HTTP"
  vpc_id   = data.aws_vpc.default.id

  stickiness {
      type = "lb_cookie"
  }
}

resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.awsalb.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type = "forward"
    forward {
      target_group {
        arn = aws_lb_target_group.origin.arn
      }

      target_group {
        arn = aws_lb_target_group.edge.arn
      }

      stickiness {
        enabled  = true
        duration = 28800
      }
    }
  }
}

I hope, this trick is helpful to you.

Edit on June 28, 2022: I used this code block on https://github.com/flightlesstux/antmedia/tree/master/aws-cluster You can check and understand how's working...

---

# Extend your ec2 Linux disk without reboot on Amazon Web Services

URL: https://ercan.cloud/extend-your-ec2-linux-disk-without-reboot-on-amazon-web-services/
Date: 2021-02-08T15:08:26+03:00

Just follow the steps and learn how to extend without reboot. Normally, It's realy hard to extend the Linux servers root disk.


For some reason, you have to extend your ec2 Linux server with our reboot and securely. In this port, you will learn how you can extend your ec2 Linux disk on aws.

In this example, you will see extend the disk size from 80GiB to 120GiB.

Step 1: Modify Volume

Login to your AWS Console and find your ec2's volume (disk). Here is the tip; in the left pane, there is an Elastic Block Store > Volumes. Right-click to your volume and then select the "Modify Volume" option. It looks like the below image. Choose your right size and click the Modify button.

Step 2: Check the disk size

First of you, you should be root or sudo in the server before run the commands. The first thing, we will check disk size, it should show the size before modify the volume. So, in this example, it shoud be a 80 GiB.

&#91;root@server ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        7.8G     0  7.8G   0% /dev
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           7.8G  416K  7.8G   1% /run
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/nvme0n1p1   80G   51G   30G  64% /
tmpfs           1.6G     0  1.6G   0% /run/user/0

Yes, it says, the whole disk size is 80 GiB, that's true. Now let's check the list of block devices. The command output should show the new disk size on /dev/nvme0n1 device.

&#91;root@server ~]# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1       259:0    0  120G  0 disk
├─nvme0n1p1   259:1    0   80G  0 part /
└─nvme0n1p128 259:2    0    1M  0 part

Yes! It says, the nvme0n1p1 is 80GiB but nvme0n1 is 120GiB. It means, we can extend our nvme0n1p1 part from 80GiB to 120GiB.

Step 3: Let's Extend

Here is the magic;

&#91;root@server ~]# growpart /dev/nvme0n1 1
CHANGED: partition=1 start=4096 old: size=167768031 end=167772127 new: size=251654111 end=251658207

The growpart command is just growed our disk partition, it doesn't save yet and now we still don't have a 120GiB disk. We should save the modification to the file system right now.

&#91;root@server ~]# xfs_growfs /
meta-data=/dev/nvme0n1p1         isize=512    agcount=41, agsize=524159 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1 spinodes=0
data     =                       bsize=4096   blocks=20971003, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 20971003 to 31456763

Step 4: Check the disk size again

Time to make sure about your new disk size. Here is the commands again for make sure.

&#91;root@server ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        7.8G     0  7.8G   0% /dev
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           7.8G  412K  7.8G   1% /run
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/nvme0n1p1  120G   51G   70G  42% /
tmpfs           1.6G     0  1.6G   0% /run/user/0

&#91;root@server ~]# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1       259:0    0  120G  0 disk
├─nvme0n1p1   259:1    0  120G  0 part /
└─nvme0n1p128 259:2    0    1M  0 part

Congrats! You successfully extended your aws ec2 Linux Server without reboot and/or lose any data on the EBS (Elastic Block Storage)

---

# Create a New Grant User on AWS RDS (MariaDB)

URL: https://ercan.cloud/create-a-new-grant-user-on-aws-rds-mariadb/
Date: 2021-01-30T23:01:56+03:00

You will learn, how to create a grant user for your RDS. It's little bit complicated process but just follow the article and know.


Today, I've played with AWS RDS with MariaDB. In the normally, AWS is created just one user for access AWS RDS. Here is the steps for create a new grant user on AWS RDS with MariaDB.

mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, BINLOG MONITOR, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER ON *.* TO 'YOUR-MYSQL-USERNAME'@'CLIENT-IP-OR-DNS-NAME' IDENTIFIED BY 'CLEAR-TEST-PASSWORD' WITH GRANT OPTION;

Query OK, 0 rows affected (0.009 sec)

This command means you successfully created your RDS user for MariaDB with all grant options and you set a password. Great. But you need to completely save this privilege. For this please run the code below.

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.009 sec)

And that's it!

P.S. If your RDS instance is public and need to access via internet, please do not forget your Security Group of your RDS Instance.

---

# Amazon S3 CORS Settings with CloudFront on Amazon Web Services

URL: https://ercan.cloud/amazon-s3-cors-settings-with-cloudfront-on-amazon-web-services/
Date: 2020-09-29T08:27:16+03:00

Learn how to set for your aws S3 Bucket CORS settings on CloudFront in Amazon Web Services. CORS settings is taking security for your bucket.


If you are serving static content directly Amazon S3 via CloudFront (CDN) you should probably get an error about fonts, images or other static contents to deliver for your users. Actually, it’s not complicated. You can fix this issue on your Amazon S3 Bucket CORS settings.

What is Amazon S3 CORS?

Cross Origin Resource Sharing (aka CORS). The CORS specification gives you the ability to build web applications that make requests to domains other than the one which supplied the primary content.

Step 1: Open your Bucket Permission settings on AWS Web Console

Visit https://s3.console.aws.amazon.com page and sign in with your credentials to your S3 console and than, select your bucket which one is connected to your CloudFront.

Step 2: Apply CORS Configuration

In Permissions tab, you can see “CORS configuration” section. Please select this section and apply the code below:

<CORSConfiguration>
 <CORSRule>
   <AllowedOrigin>https://YOURWEBSITEADDRESS.com</AllowedOrigin>
   <AllowedMethod>PUT</AllowedMethod>
   <AllowedMethod>POST</AllowedMethod>
   <AllowedHeader>*</AllowedHeader>
 </CORSRule>
 <CORSRule>
   <AllowedOrigin>*</AllowedOrigin>
   <AllowedMethod>GET</AllowedMethod>
 </CORSRule>
</CORSConfiguration>

After the paste, change the <AllowedOrigin></AllowedOrigin> with your website address and click the “Save” button.

Conclusion

Hooray! Now, you have successfully set your CORS settings and now your users can able to view your content which one is located in your Amazon S3 Bucket.

---

# Take your GitLab backup everyday if it works in Docker

URL: https://ercan.cloud/take-your-gitlab-backup-everyday-if-it-works-in-docker/
Date: 2020-09-16T15:27:44+03:00

Learn how to automate your Docker GitLab backup. You should make a backup for your repository security. Let's automate the process...


I'm using GitLab in docker (because I love containerization) and I'm really okay with that because I can move the GitLab environment everywhere if I need and I can feel more elastic when I'm using docker.

GitLab backups are really so important because you are keeping all of the source code and maybe wiki maybe more than all of the source code and backup is the first thing if you have a service running at a critical level.

Make sure about your docker run command

I run my GitLab in docker with these command set:

&#91;ercan@gitlab ~]$ docker run --detach \
  --hostname gitlab.ercanermis.com \
  --publish 443:443 --publish 80:80 --publish 22:22 \
  --name gitlab \
  --restart always \
  --volume /srv/gitlab/config:/etc/gitlab \
  --volume /srv/gitlab/logs:/var/log/gitlab \
  --volume /srv/gitlab/data:/var/opt/gitlab \
  gitlab/gitlab-ce:latest

First of all, we need to get to know what's the docker container name of the GitLab. In my setup, I'm using GitLab Community Edition and I gave the name as a "gitlab" for the docker. If you don't know your gitlab docker's name, there is an easy way to learn what it is.

&#91;ercan@gitlab ~]$ docker ps -a | grep gitlab-ce

The last column shows the docker container name. Here is the output:

9c787206fbda gitlab/gitlab-ce:latest "/assets/wrapper" 8 hours ago Up 8 hours (healthy) 0.0.0.0:22->22/tcp, 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp gitlab

Take a backup and know the backup path

Actually GitLab backup is really easy. Here is the command for the trigger backup mechanism of GitLab.

docker exec gitlab gitlab-rake gitlab:backup:create DIRECTORY=gitlab

Trigger command isn't enough because you should take a backup of gitlab.rb and gitlab-secrets.json files are located in /srv/gitlab/config/ path. I prefer to copy (with cp command) these files to /srv/gitlab/data/backups/ path because gitlab-rake is exporting to the same directory.

My GitLab Docker Backup backup bash script looks like this:

#!/bin/bash
#Author: Ercan Ermis - https://ercanermis.com

docker exec gitlab gitlab-rake gitlab:backup:create DIRECTORY=gitlab
cp /srv/gitlab/config/gitlab.rb /srv/gitlab/data/backups/$(date +%F_%H-%M)_gitlab.rb
cp /srv/gitlab/config/gitlab-secrets.json /srv/gitlab/data/backups/$(date +%F_%H-%M)_gitlab-secrets.json

Which files do you have after run backup bash?

gitlab is generating the backup file name like1600262658_2020_09_16_13.3.6_gitlab_backup.tar It means epochtime_year_month_day_gitlabVersion_gitlab_backup.tar That's the reason why I use the $date function when copying gitlab.rb and gitlab-secrets.json to the /srv/gitlab/data/backups/ folder.

I'm using aws S3 service to keeping my backups securely and redundant and when the backup is completed and ready in /srv/gitlab/data/backups/ folder, I'm syncing backed up files to s3 with aws s3 sync /srv/gitlab/data/backups/ s3://ercanermis-backup/gitlab/ command and deleting the backup file on the server.

You can also use your own solution to keep your backup redundant and securely like SMB, sFTP or another block storage solution from another provider.

Final: Let's Automate Backup

Here is the final bash for automating GitLab Docker Backup located in /opt/gitlab-backup.sh

#!/bin/bash
#Author: Ercan Ermis - https://ercanermis.com

docker exec gitlab gitlab-rake gitlab:backup:create DIRECTORY=gitlab
cp /srv/gitlab/config/gitlab.rb /srv/gitlab/data/backups/$(date +%F_%H-%M)_gitlab.rb
cp /srv/gitlab/config/gitlab-secrets.json /srv/gitlab/data/backups/$(date +%F_%H-%M)_gitlab-secrets.json
aws s3 sync /srv/gitlab/data/backups/ s3://ercanermis-backup/gitlab/ && rm -rf /srv/gitlab/data/backups/*

After create your .sh file, you can use your crontab for trigger run your own /opt/gitlab-backup.sh

---

# Find large files in CentOS, ubuntu and MacOS easily

URL: https://ercan.cloud/find-large-files-in-centos-ubuntu-and-macos-easily/
Date: 2020-07-18T13:08:30+03:00

Sometimes, it's really hard to find the big files path Linux. In this tutorial, you will learn how you can find it easily.


Sometimes, maybe everybody needs to check large files in the system. There some useful commands which I'm using often. Find the big files in Linux easily.

Here is the first pattern:

find /home/ercan/ -type f -size +100000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

If file size more than 250 MB in the system:

sudo find /home/ercan/ -type f -size +250000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

Here is the results:

/home/ercan/.local/share/Steam/steamapps/common/Counter-Strike: 631M
/home/ercan/.local/share/Steam/steamapps/common/Counter-Strike: 248M
/home/ercan/.local/share/Steam/steamapps/common/Counter-Strike: 251M
/home/ercan/.local/share/Steam/steamapps/common/Counter-Strike: 464M
/home/ercan/.local/share/Steam/steamapps/common/Counter-Strike: 306M
/home/ercan/Downloads/CentOS-7-x86_64-Minimal-2003.iso: 1.1G
/home/ercan/VirtualBox: 13G

---

# Fix "Error: rpmdb open failed" on CentOS or Amazon Linux 2

URL: https://ercan.cloud/fix-error-rpmdb-open-failed-on-centos-or-amazon-linux-2/
Date: 2020-04-20T11:49:37+03:00

If you having rpmdb issues with your yum, you can easily follow the same steps and fix your package manager in a minute.


Yesterday, I have started to test some services on running Amazon Linux 2 powered by Amazon Web Services cloud. For some reason, my package manager yum is broken and when I started to update my yum database, I've got an error about rpmdb open failed. You can see my error below. If you are running on Centos7 or another Linux distro using the package manager as yum, you can also continue to read.

error: rpmdb: BDB0113 Thread/process 18275/140171317404864 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:

Error: rpmdb open failed

I decided to move all rpm database packages to /tmp directory.

sudo mv /var/lib/rpm/__db* /tmp

After this movement, yum should be clean completely. I run "yum clean all" to clean my yum database.

sudo yum clean all

After the cleaning process, yum said to me, cleaning is done.

Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Repository elastic-7.x is listed more than once in the configuration
Bad id for repo: root@ip-11-12-30-149 yum.repos.d, byte = @ 4
Cleaning repos: amzn2-core amzn2extra-epel elastic-7.x epel treasuredata
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos

I crossed my fingers and tried to update my yum database with the latest versions of packages. For this; I run "sudo yum update" command.

sudo yum update

Now, yum is running perfectly again!

Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Repository elastic-7.x is listed more than once in the configuration
Bad id for repo: root@ip-11-12-30-149 yum.repos.d, byte = @ 4
amzn2-core                                                                      | 2.4 kB  00:00:00
amzn2extra-epel                                                                 | 1.3 kB  00:00:00
elastic-7.x                                                                     | 1.3 kB  00:00:00
epel/x86_64/metalink                                                            |  21 kB  00:00:00
epel                                                                            | 4.7 kB  00:00:00
treasuredata                                                                    | 2.9 kB  00:00:00
(1/9): amzn2-core/2/x86_64/group_gz                                             | 2.5 kB  00:00:00
(2/9): amzn2-core/2/x86_64/updateinfo                                           | 200 kB  00:00:00
(3/9): amzn2extra-epel/2/x86_64/primary_db                                      | 1.8 kB  00:00:00
(4/9): elastic-7.x/primary                                                      | 115 kB  00:00:00
(5/9): epel/x86_64/group_gz                                                     |  95 kB  00:00:00
(6/9): epel/x86_64/updateinfo                                                   | 1.0 MB  00:00:00
(7/9): epel/x86_64/primary_db                                                   | 6.8 MB  00:00:00
(8/9): amzn2-core/2/x86_64/primary_db                                           |  39 MB  00:00:00
(9/9): treasuredata/2/x86_64/primary_db                                         |  15 kB  00:00:00
elastic-7.x                                                                                315/315
192 packages excluded due to repository priority protections
No packages marked for update

If you having rpmdb issues with your yum, you can easily follow the same steps and fix your package manager in a minute.

---

# Error: No space left on the device when starting/stopping services only

URL: https://ercan.cloud/error-no-space-left-on-the-device-when-starting-stopping-services-only/
Date: 2020-03-01T07:46:00+03:00

Learn how to fix "No space left on the device" error when starting and/or stopping services only on Amazon Linux 2


I'm using Amazon SSM Agent for connecting to EC2 instance via securely. Amazon SSM Agent is also able to connect to the EC2 instance console via aws.amazon.com web console. So, this is my preferred way.

About a few days ago, when I tried to connect to EC2 via Amazon SSM Agent it doesn't respond and after that, I decided to connect via a regular way, yes with SSH.

Of course, first of all, I want to restart Amazon SSM Agent and I see this:

[root@i-0cd9514c60d532e78 ~] systemctl restart amazon-ssm-agent.service
Error: No space left on device

I didn't wait for this error message about disk status. And want to check disk usage. Now, I see this:

[root@i-0cd9514c60d532e78 ~] df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 960M 0 960M 0% /dev
tmpfs 978M 0 978M 0% /dev/shm
tmpfs 978M 420K 978M 1% /run
tmpfs 978M 0 978M 0% /sys/fs/cgroup
/dev/nvme0n1p1 8.0G 2.6G 5.5G 32% /
tmpfs 196M 0 196M 0% /run/user/1000

Oops! It looks like, this error message is not about disk usage but I have to make sure. Now, time is checking inodes.

[root@i-0cd9514c60d532e78 ~] df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
devtmpfs 245750 304 245446 1% /dev
tmpfs 250254 2 250252 1% /dev/shm
tmpfs 250254 388 249866 1% /run
tmpfs 250254 16 250238 1% /sys/fs/cgroup
/dev/nvme0n1p1 4193216 67672 4125544 2% /
tmpfs 250254 1 250253 1% /run/user/1000

Now that I could confirm it is not inodes too I did further research and I found there exist known bug reports for this same issue where systemctl action of "start", "stop" and "reload" will show a "No space left on device" error, but actions of "enable", "disable" and "kill" do not. In the bugs, it is reported that this error occurs is a result of the inotify "max_user_watches" limit. inotify has a limit on the number of files and directories it can monitor for changes. To silence the error you need to increase the value of "/proc/sys/fs/inotify/max_user_watches" to allow more files and directories to be added and monitored for changes.

To change the value in realtime you can run: 

echo 1048576 > /proc/sys/fs/inotify/max_user_watches

To persistently loads this same increased value after a reboot or stop/start action is performed:

Add line "fs.inotify.max_user_watches=1048576" to /etc/sysctl.conf You can check your sysctl configuration with sysctl -p command.

References:
https://bugzilla.redhat.com/show_bug.cgi?id=894483 
https://bugzilla.redhat.com/show_bug.cgi?id=1452933 

Conclusion

---

# Juniper SRX110H-VA VDSL2 Configuration Step by Step

URL: https://ercan.cloud/juniper-srx110h-va-vdsl2-configuration-step-by-step/
Date: 2020-02-09T06:48:00+03:00

Set a VDSL2 connection for Juniper SRX-110 easily. I've already created a main configration and you can use if you need. Just edit few lines..


I was talking on the phone with Kenan Bilgic last night and I told him to "I want to get a Juniper SRX-110H-VA". After this, Kenan told me, "I've already got a Juniper SRX-110H-VA, and just send me to your address, I can ship it for you."

I'm using TurkNet as an internet provider and they are really cheap and fast internet provider in my neighborhood. My connection type is VDSL2 and I need to make a VPN connection through Amazon Web Services (AWS).

My router has arrived!

I'm so excited about my new router device. I need to configure it before connecting to the copper phone cable via RJ13. I configured my router as a basic and I pushed my config to my GitHub repository.

I used JUNOS 12.1X46-D86 software version. This version is the latest supported version for Juniper SRX110H-VA. 

Before your start

 vLAN ID (Important) Every ISPs are using their own vLAN ID for the connection. In Turkey, ISPs are using vlan-id35.

 
Username You should know your username for the connection which one is ISP provided.

 
Password As like username but this time it's your password. This is also ISP provided.

 
Terminal Cable If you SRX110H-VA ssh service is disabled, you need to terminal cable for load a configuration. If you don't have you can buy from internet.

 

You can find more details on the GitHub repository. Here is the link; https://github.com/flightlesstux/juniper-srx-config/tree/master/srx110h-va

After the steps, I see my Juniper is connected to the internet, yay!

root@juniper-srx110> show interfaces pp0.0 extensive | no-more
  Logical interface pp0.0 (Index 81) (SNMP ifIndex 534) (Generation 146)
    Flags: Point-To-Point SNMP-Traps 0x0 Encapsulation: PPPoE
    PPPoE:
      State: SessionUp, Session ID: 1,
      Session AC name: TT-35-IZMIR-ERC-SSR-01, Remote MAC address: ac:60:b6:5f:b7:bc,
      Configured AC name: None, Service name: TurkNet VDSL2,
      Auto-reconnect timeout: 5 seconds, Idle timeout: Never,
      Underlying interface: pt-1/0/0.0 (Index 80)
    Traffic statistics:
     Input  bytes  :           5396015635
     Output bytes  :           2818762178
     Input  packets:              4164167
     Output packets:             13992298
    Local statistics:
     Input  bytes  :               928428
     Output bytes  :               944238
     Input  packets:                25117
     Output packets:                25139
    Transit statistics:
     Input  bytes  :           5395087207                 1912 bps
     Output bytes  :           2817817940                 3472 bps
     Input  packets:              4139050                    3 pps
     Output packets:             13967159                    5 pps
  Keepalive settings: Interval 10 seconds, Up-count 1, Down-count 3
  Keepalive statistics:
    Input : 3499 (last seen 00:00:37 ago)
    Output: 21028 (last sent 00:00:09 ago)
  LCP state: Opened
  NCP state: inet: Opened, inet6: Not-configured, iso: Not-configured, mpls: Not-configured
  CHAP state: Closed
  PAP state: Success
    Security: Zone: untrust
    Flow Statistics :
    Flow Input statistics :
      Self packets :                     579
      ICMP packets :                     620
      VPN packets :                      0
      Multicast packets :                0
      Bytes permitted by policy :        41725273176
      Connections established :          0
    Flow Output statistics:
      Multicast packets :                0
      Bytes permitted by policy :        2824110282
    Flow error statistics (Packets dropped due to):
      Address spoofing:                  0
      Authentication failed:             0
      Incoming NAT errors:               0
      Invalid zone received packet:      0
      Multiple user authentications:     0
      Multiple incoming NAT:             0
      No parent for a gate:              0
      No one interested in self packets: 0
      No minor session:                  0
      No more sessions:                  0
      No NAT gate:                       0
      No route present:                  0
      No SA for incoming SPI:            0
      No tunnel found:                   0
      No session for a gate:             0
      No zone or NULL zone binding       0
      Policy denied:                     0
      Security association not active:   0
      TCP sequence number out of window: 45
      Syn-attack protection:             0
      User authentication errors:        0
    Protocol inet, MTU: 1492, Generation: 159, Route table: 0
      Flags: Sendbcast-pkt-to-re, Negotiate-Address
      Addresses, Flags: Kernel Is-Preferred Is-Primary
        Destination: 172.17.1.89, Local: 100.84.25.149, Broadcast: Unspecified, Generation: 154

Previously, I and Kenan worked in the same company and we really like each other. I'm really appreciated Kenan's favor!

---

# Enable Logrotation for Docker Containers

URL: https://ercan.cloud/enable-logrotation-for-docker-containers/
Date: 2019-11-29T17:17:00+03:00

Docker on your system, the container running in the docker can generate a large number of logs while you are not aware of the background.


When you install and use Docker on your system, the container running in the docker can generate a large number of logs while you are not aware of the background.

Although the Docker service normally supports logrotate, this support comes off by default. If one day your disk ends suddenly, then you will know exactly what I mean. : )

If you are installing the docker service for the first time in a server and running the container for the first time, you can follow the steps below.

When the docker service is in a stop state, first open the file that the docker service will need for the logrotate operation with the text editor. My favorite is vim, I used vim. You can also choose another editor like nano, pico.

sudo vim /etc/docker/daemon.json

Then add the following lines in this file and save and close the file.

{
 "log-driver": "json-file",
 "log-opts": {
     "max-size": "10m",
     "max-file": "3"
     }
 }

Now, don't forget to restart your docker service.

sudo systemctl restart docker

Now, your docker service started to rotation logs under your "/var/lib/docker/containers/CONTAINERID/" path.

---

# Download specific file extension via wget easily on terminal

URL: https://ercan.cloud/download-specific-file-extension-via-wget-easily-on-terminal/
Date: 2019-11-22T16:37:00+03:00

Sometimes, there may be files with more than one extension in a directory or folder. For example, .jpg, .png, .gif, .mp3, .mp4 etc.


Sometimes, there may be files with more than one extension in a directory or folder. For example, .jpg, .png, .gif, .mp3, .mp4 etc. You want to download only gif files specifically from that source, but don't know how to do this? Don't be afraid, you're right.

wget is small software with a GNU license (i.e. free), by default on Linux and macOS operating systems.

The other day, I wanted to download the same .mp4 and .webm videos in an ftp directory, but I would either download the .mp4 ones or .webm files. Because of the size of 100 .mp4 files is 10 GB, 100 .webm files with the same content are only 6 GB. I decided to download the .webm files as I wanted to allocate much less disk space.

For this, I ran the following command and started watching the screen.

wget -r -A .webm http://videos.example.com/u/source/

Yes, as you can see, that's all! If I wanted to download .mp4 files from this address, I would have to run this command this time;

wget -r -A .mp4 http://videos.example.com/u/source/

This command also supports https or ftp instead of http. So you can use http: //, https: // or ftp: // as the connection protocol.

Conclusion

In this article, we have learned how to recursive download files with specific extensions using wget. I hope it worked for you. You can use this command on CentOS, ubuntu, Amazon Linux, macOS, and many other Linux distros.

---

# Find the exact size of certain files in Linux via terminal

URL: https://ercan.cloud/find-the-exact-size-of-certain-files-in-linux-via-terminal/
Date: 2019-11-15T19:00:00+03:00

You may be wondering about the size of these files or just want to see the size of the files with the corresponding extension for a ...


One day, there may be a lot of files in more than one extension in the macOS, Linux or server you use, and you may be wondering about the size of these files or just want to see the size of the files with the corresponding extension for a specific study.

The other day, the server I was working on had both * .webm and * .mp4 file extensions, and I had to upgrade the server's disk, or I would delete those with the same .mp4 extension from the same videos.

There are multiple ways to do this. I will only describe three different and most frequently used commands.

For example, if there is no hidden folder in the directory you are in, you can learn the size of .jpg files in that directory with the simplest method with the following command.

First of all, since I want to list the jpg files that exist in my directory and see one, I run the command below and display its output. Let's check my Documents folder on my Mac, what I have.

➜  Documents ls -la | grep jpg
-rw-r--r--@  1 ercanermis  staff  1607683 Feb 26 10:14 wallpaper-for-vertical-monitor.jpg
-rw-r--r--@  1 ercanermis  staff   242165 Aug 19  2019 10155854826986546_245396163509039923.jpg
-rw-r--r--@  1 ercanermis  staff   272199 Feb 26 10:00 C2014_Q2.jpg

Then what are the sizes of these jpg files? For the answer to the question, the following command is sufficient.

➜  Documents du -ch *.jpg
1.5M	wallpaper-for-vertical-monitor.jpg
240K	10155854826986546_245396163509039923.jpg
268K	C2014_Q2.jpg
2.0M	total

If the total size of these files is important to you, you can do as follows;

➜  Documents du -ch *.jpg | grep total
2.0M	total

Is it really 3 jpg files into my Documents folder?

I don't think so and I decided to check what I have inside. Because I will have some hidden files or folders. Let's check deeper what are the sizes of all jpg files?

➜  Documents find . -type f -name '*.jpg' -exec du -ch {} + | grep total
4.6M	total

Hmm. Now, it says 4.6M total but previously, I mean "du -ch *.jpg | grep total" command output was 2.0M. How it possible? Because I have hidden files and folders. Let's check them.

➜  Documents find . -type f -name '*.jpg' -exec du -ch {} +
1.5M	./free-wallpaper-for-vertical-monitor.jpg
268K	./C2014_Q2.jpg
240K	./10155854826986546_245396163509039923.jpg
372K	./git/Resources/main/1/std-front.jpg
324K	./git/Resources/main/3/std-front.jpg
272K	./git/Resources/main/2/std-front.jpg
696K	./git/Resources/main/splash_background.jpg
4.0K	./git/Resources/popup/social/bg-description.jpg
 40K	./git/Resources/popup/news/background.jpg
 40K	./git/Resources/popup/rooms/bg.jpg
 80K	./git/Resources/popup/bundle-bg-2.jpg
 88K	./git/Resources/popup/bundle-bg-3.jpg
 88K	./git/Resources/popup/bundle-bg-1.jpg
 96K	./git/Resources/popup/bundle-bg-5.jpg
4.0K	./git/Resources/popup/eventLeftScroll.jpg
 92K	./git/Resources/popup/eventBg.jpg
 40K	./git/Resources/popup/progression/upperBg.jpg
8.0K	./git/Other/nd_gcsdk_image_default.jpg
4.0K	./git/Other/nd_gcsdk_loading_bg.jpg
 20K	./git/Other/docs/img.jpg
 84K	./git/Other/docs/framework_architecture.jpg
 44K	./Do_My_Verification/1.jpg
264K	./Do_My_Verification/2.jpg
4.6M	total

There you go! I forgot my git folder and I have some jpg files into it!

Conclusion

In two different ways, you learned to check the specific file extensions in a folder and list their size. You can run these commands on CentOS, ubuntu or macOS operating systems.

It will also work on different Linux distributions, such as Raspberry Pi.

---

# Disable SELinux on CentOS 7 or CentOS 8

URL: https://ercan.cloud/disable-selinux-on-centos-7-or-centos-8/
Date: 2019-11-08T19:00:00+03:00

This tutorial shows you how you can disable SELinux on your CentOS server. The CentOS version should be CentOS 7 or CentOS 8.


This tutorial shows you how you can disable SELinux (Security-Enhanced Linux) on your CentOS server. The CentOS version should be 7 or 8. The version does not matter.

What is SELinux?

SELinux is a security mechanism directly controlling by the kernel. It allows administrators and users more control over access controls on access based SELinux policies.

SELinux has three different modes of operation. Here they are:

Enforcing: Allows access based on SELinux policies and the policy rules.
Permissive: SELinux only logs actions that would have been denied if running in enforcing mode. 
Disabled: No messages logged and there is no more SELinux policy enabled on the server. Most of the time disabled mode is using web control panels like cPanel and/or Plesk.

Prerequisites

Only the root user or a user with sudo privileges can update SELinux mode.

Check the SELinux Mode

You can make sure about your SELinux status with the "sestatus" command. The print shows that SELinux is enabled and mode set to enforcing.

SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      31

Disabling SELinux

I strongly recommended changing the mode to permissive. Sometimes some of the applications don't like SELinux and they required to mode disabled.

To permanently change SELinux mode from enforcing to disabled, first of all, you should use your text editor like vi or nano.

Open the /etc/selinux/config file and change the SELINUX value to disabled on the 6th line like below:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#  enforcing - SELinux security policy is enforced.
#  permissive - SELinux prints warnings instead of enforcing.
#  disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
# targeted - Targeted processes are protected,
#  minimum - Modification of targeted policy. Only selected processes are protected. 
#  mls - Multi Level Security protection.
SELINUXTYPE=targeted

Now you can save the file and exit from the text editor. Time to reboot the server:

sudo reboot

Final Control of SELinux Status

Now, we should make sure about SELinux successfully disabled on the server.  Run "sestatus" command again, the output should like this: 

SELinux status:                 disabled

Conclusion

Now you can change the SELinux modes which one is suitable for you or you need. SELinux is a security mechanism by implementing mandatory access control (MAC). SELinux comes enforcing mode by default on CentOS7 and CentOS8 system. It can be modifying by editing the configuration file and rebooting the server.

You can access more information about SELinux on their CentOS SELinux pages.

---

# Hello Blog!

URL: https://ercan.cloud/hello-blog/
Date: 2019-11-03T22:24:00+03:00

My blog address contains my first name and my last name. So, you know my name I guess : )


I don't know how many times I set up a blog to myself to take notes or share some technical information about Linux servers and some Linux software such as nginx, phpMyAdmin, MySQL, PHP-FPM, etc but I never success about writing a blog post regularly.

My blog address contains my first name and my last name. So, you know my name I guess : )

I'm working at Masomo as a Linux System Administrator but mostly doing DevOps things with an awesome team.

Every day I'm solving problems in the office and using a lot of different technologies to make a perfect and zero-problem mobile game all over the world. It's Head Ball 2 Mobile Game.

Head Ball 2 from Masomo

In my mind, here are the categories for my future posts.

aws (Amazon Web Services)
DevOps Tools and DevOps Culture
Docker
Kubernetes
Linux Server (CentOS, Debian, ubuntu)

In the end, I want to write about my experiences and my know-how in this blog. I hope, this time I'll be successful and keep this blog up-to-date.

Last of all, if you are interested in what's next on this blog, you can feel free to create an e-mail subscription on the right side of the newsletter area. Thank you for visiting.


---

# AWS Multi-party Approval for Organizations

URL: https://ercan.cloud/aws-organizations-multi-party-approval/

Multi-party approval is a capability of AWS Organizations that blocks a predefined set of high-risk operations until a quorum of named humans approves them out of band. A single set of credentials, even a valid one, can no longer pull the trigger alone. The requester starts the action, a separate approval team reviews it in a dedicated portal, and the operation only runs once the team's approval threshold is met.

This is the "four eyes" principle enforced at the API layer instead of in a runbook or a Slack message. It closes the gap between "we have a policy that says two people must sign off" and "the platform physically refuses to proceed without two sign-offs."

What it actually is

Multi-party approval (MPA) sits inside AWS Organizations and depends on IAM Identity Center for the human identities. You define an approval team, a set of approvers drawn from your identity source, and you attach an approval threshold: the minimum number of those people who must say yes before a protected operation can run. You then mark specific operations as protected. From that point on, anyone who invokes a protected operation does not get an immediate result. They get a pending request. MPA emails every approver on the team, each approver opens the approval portal, reviews the request, and approves or denies it. Once the threshold is reached, the original caller's operation is allowed to proceed.

The vocabulary: Approval team is a named group of approvers, backed by IAM Identity Center users. Approval threshold is how many approvers must approve. Protected operation is the specific action you have placed behind the gate. Approval portal is the out-of-band surface where approvers review and respond, deliberately separate from the console session that made the request.

How the flow works

Three steps. 1) Request: a user or automated caller invokes a protected operation; instead of executing, AWS creates an approval session and notifies the team. 2) Approval session: approvers receive an email with a link, open the portal, see exactly what is being requested, by whom, and against which resource, and approve or deny independently. 3) Execution: when approvals reach the threshold, the operation runs; if the session times out or the threshold is never met, the operation never happens. The requester cannot approve their own request from inside their own session, which is what makes a stolen session token or a compromised CI role insufficient on its own.

Why it matters for security

It removes the single point of compromise: for the handful of operations that can ruin your week, MPA turns "one compromised credential" into "you also need to subvert N independent humans at the same time." It enforces separation of duties at the API, moving the rule from a compliance document into the control plane. It is a brake against ransomware and destructive insiders: if the operations that delete or alter your protected recovery points sit behind a quorum, an attacker who owns one identity cannot quietly destroy your last line of recovery. It produces an audit trail you can defend for SOC 2, ISO 27001, PCI DSS, or internal post-incident review.

Where it fits, and where it does not

Use it when the operation is rare, high blast-radius, and irreversible, when you run AWS Organizations with IAM Identity Center, when you need distributed sign-off for compliance or Zero Trust, and when you want to protect deletion or alteration of last-resort backups. Skip it when the operation is frequent and needs immediate execution, when you run standalone accounts with no Organizations or Identity Center, when the risk does not justify the overhead, or when you need unattended automation with no human in the loop. MPA is a scalpel, not a blanket.

Failure modes to plan for: approver availability (size the team larger than the threshold), break-glass tension (decide and rehearse how you reach approvers out of hours), approval fatigue (keep the protected list short), and identity source dependency (MPA leans on IAM Identity Center, so it belongs in your resilience planning).

The takeaway: Multi-party approval is one of the few AWS controls that changes the math for an attacker rather than just raising a flag. Apply it narrowly, staff the approval teams properly, and rehearse the break-glass path.


---

# No Phone, No Account: Google's QR Wall and GDPR

URL: https://ercan.cloud/google-qr-phone-verification-gdpr/

Try to create a Google account today and you may hit a wall before you type a single character of your name. The screen says "Verify some info before creating an account", shows a QR code, and tells you to scan it with your phone's camera. There is no link to click, no email fallback, no skip path. No smartphone with a working camera means no account. That quietly turns a phone into a precondition for using the web. Written from the Netherlands, inside the EU, this looks less like a security feature and more like mandatory identity collection dressed up as anti-abuse.

What the screen actually demands

To create an account you are asked to own a smartphone with a camera, hand the flow to that phone and complete steps not shown up front, tie the account to a phone number (in most of the EU tied to verified government identity at the SIM level), and later provide a date of birth. None of that is a technical requirement for an email account to function. These are collection decisions, not engineering constraints.

Anti-abuse is doing a lot of work here

The justification is bot prevention, which is real, and rate-limiting account creation is legitimate. But the control does not ask you to solve something a human can and a script cannot. It asks you to present a phone. The thing verified is not "are you human", it is "can you produce a device and number we can fingerprint and correlate." Numbers added for security have a long history of drifting into ad targeting and cross-service correlation. Security teams call this function creep; GDPR calls it a purpose limitation problem.

Where this collides with GDPR

Data minimisation, Article 5(1)(c): personal data must be adequate, relevant and limited to what is necessary. A phone number and date of birth are not necessary to operate a mailbox, so the burden is on the controller to justify collection, not on the user to justify refusal. Freely given consent, Articles 4(11) and 7(4): consent is valid only if freely given, and you must consider whether a service is made conditional on processing that is not necessary. "No phone number, no account" is the textbook definition of conditional. Proportionality and necessity: even under legitimate interest, forcing every user through hardware they may not own, to defeat a fraction of signups, is hard to call the least intrusive option when CAPTCHA, email confirmation, and behavioural signals exist and need no camera.

The exclusion nobody designs for

A QR-and-camera gate assumes a recent smartphone, the ability to see and scan a code, and comfort moving signup onto a handset. It excludes desktop-only users, people with visual impairments, feature-phone users, people with a minimal device footprint, and anyone who does not want a personal phone wired into every login. Accessibility and data protection point the same direction: a central service should have a path that does not require owning particular hardware.

What the EU could actually do about it

The GDPR machinery already exists; it needs enforcement, not new law. Treat phone number and date of birth at signup as non-necessary by default and require proof of necessity. Mandate a genuine alternative path that does not require a smartphone or number, on Article 7(4) conditionality logic. Scrutinise purpose: a number collected for anti-abuse and later used for advertising is a purpose-limitation breach regardless of the signup wording. The cookie-wall rulings set the precedent: "consent or leave" is not real consent, and a hardware wall is the same argument with a camera attached.

What you can do today

Use a provider that does not demand a phone for the accounts you control; several EU-based mail providers treat a number as optional. Keep identity and convenience separate, and do not reuse the number that anchors your banking and government identity. Exercise your rights: a Subject Access Request and, where appropriate, a complaint to your national DPA are the mechanism that turns principle into pressure.

The takeaway: A QR-and-phone wall at account creation is sold as security, but the data it secures is yours, handed over as the price of entry. In the EU that price has a legal frame: data must be minimised, consent must be free, and a service cannot quietly make itself conditional on information it does not need to function. Whether a regulator says so out loud is a matter of will rather than law.


---

# AWS Lambda MicroVMs: Serverless Gets a Stateful, Isolated Sandbox

URL: https://ercan.cloud/aws-lambda-microvms-serverless-sandboxes/

AWS Lambda MicroVMs are a new serverless compute primitive that gives you virtual-machine-level isolation, near-instant launch from a snapshot, and stateful execution you can suspend and resume. It runs on Firecracker, the same lightweight virtualization that already backs more than 15 trillion Lambda invocations a month. Lambda just grew the ability to hand each session its own dedicated, isolated, long-lived sandbox without you managing any virtualization yourself. The target use cases are multi-tenant code execution, interactive notebooks, vulnerability scanners, AI coding assistants: anywhere you must safely run code your own application did not write.

What is actually new

Regular Lambda functions are stateless, short-lived, and event-driven. MicroVMs are a different primitive inside the same service for when you need a session that lives for minutes or hours, keeps state in memory, and isolates untrusted code at the kernel level. AWS frames it as three capabilities no single AWS compute service offered together before:

1. VM-level isolation. Each session runs in its own dedicated Firecracker MicroVM with no shared kernel and no shared resources between users. Untrusted code from one tenant is contained to its own environment, with no path to other environments or the host. Containers share a kernel; a MicroVM does not.

2. Snapshot-fast launch and resume. The model is image-then-launch. You build a MicroVM Image from a Dockerfile plus a code artifact in Amazon S3. Lambda runs the Dockerfile, initializes your application, and takes a Firecracker snapshot of the running memory and disk state. Every MicroVM launched from that image resumes from the pre-initialized snapshot instead of cold-booting. Even a multi-gigabyte session comes back fast. This is the SnapStart idea taken to its end: you restore a running process, not start one.

3. Stateful, suspendable execution. A running MicroVM keeps memory, disk, and processes across the session. When idle it can be suspended with state intact, then resumed when traffic returns. Installed packages, loaded data, and working files are still there. MicroVMs support up to 8 hours of total runtime and auto-suspend after a configurable idle window.

How to operate it

Two CLI calls carry most of the workflow. aws lambda-microvms create-microvm-image (with --code-artifact uri, --name, --base-image-arn arn:aws:lambda:...:microvm-image:al2023-1, --build-role-arn) builds the image; build logs stream to CloudWatch under /aws/lambda/microvms/<image-name> and the image gets its own ARN and version. aws lambda-microvms run-microvm (with --image-identifier, --execution-role-arn, --idle-policy '{"maxIdleDurationSeconds":...,"suspendedDurationSeconds":...,"autoResumeEnabled":true}') launches a MicroVM, returning a unique ID and a dedicated endpoint URL with the app already running from the snapshot. No networking to configure. The idle-policy is where lifecycle control lives.

Where it fits, and where it does not

Reach for MicroVMs when you run untrusted or user-supplied code needing kernel-level isolation, when a session must hold state across minutes or hours, when you want per-user dedicated environments without running the fleet yourself, or when idle periods are normal and you want suspend-resume instead of paying to idle. Stay on regular Lambda when code is trusted and the work is a short stateless transform. MicroVMs are not a replacement for functions and not a cheap way to run a 24/7 server: the 8-hour ceiling and suspend-resume model are built for sessions. A constantly-busy long-running service is still ECS or EC2 territory.

The snapshot caveat

Because every MicroVM resumes from a pre-initialized snapshot, anything your app does during initialization that assumes freshness can surprise you: unique IDs, network connections, random seeds, or ephemeral data loaded once can be frozen into the snapshot and reused. Same class of gotcha SnapStart users know: if it must be unique or fresh per session, generate it after resume, not during init. AWS exposes hooks for this.

The takeaway: Lambda MicroVMs are the clearest signal yet that serverless is done being only stateless and short-lived. A managed Firecracker sandbox with real isolation, snapshot-speed starts, and suspend-resume state, exposed as two API calls and an idle policy. For multi-tenant code execution it removes the hardest, least differentiated part of the job.

# AZ Isolation Held. Your Architecture Didn't.

URL: https://ercan.cloud/nothing-fails-alone-part-1-az-isolation-held/
Date: 2026-06-03T09:00:00+02:00

On the evening of May 7, cooling failed in one data hall of one availability zone in us-east-1, racks lost power, and AWS's blast radius did what the documentation promises: the damage stayed inside use1-az4. Every other zone kept serving. The companies that went down that night went down inside their own architecture, and we know that with unusual precision because Coinbase published a postmortem on Monday that says so in its own words. Trading was, in their phrasing, "unavailable or degraded for roughly eight hours, with full recovery of all systems taking another twelve", during an event in which the platform's zone boundary worked.

This is Part 1 of Nothing Fails Alone, a series about the gap between the availability you buy from a cloud provider and the availability you actually build on top of it. Each part takes one real incident, extracts the architectural claim it tests, and ships a small lab you can run against your own account. The labs live in a companion repo, nothing-fails-alone on GitHub (https://github.com/flightlesstux/nothing-fails-alone), and every one of them is read-only.

What actually happened in use1-az4

The first incident report on the AWS Health Dashboard was timestamped 5:25 PM PDT on May 7: problems in the use1-az4 availability zone of us-east-1. A follow-up update stated the mechanism plainly: "EC2 instances and EBS volumes hosted on impacted hardware are affected by the loss of power during the thermal event." By 6:47 PM PDT, AWS said it was working to bring temperatures back to normal levels and warned that other services depending on the affected EC2 instances and EBS volumes in that zone could also be impaired.

Coinbase's postmortem fills in the physical cause: multiple chiller units failed at the same time in a single data hall, and the cooling loss triggered a thermal-safety shutdown of the affected racks. That is what "loss of power during the thermal event" means in practice. The racks were not destroyed; they were deliberately powered down because running compute without cooling destroys it. Recovery was gradual: at 10:11 AM PDT on May 8, AWS reported additional cooling capacity online and some racks recovered, and cooling capacity was back to pre-incident levels at 1:50 PM PDT on May 8, roughly twenty hours after the first report. A thermal event is measured in hours because physics is, and racks come back rack by rack.

What did not happen

The other availability zones in us-east-1 stayed healthy, which is the part AWS actually promised: availability zones are physically separate facilities with independent power and cooling, and that independence was tested by a real facility failure and held. The one honest caveat: AWS shifted traffic away from the stricken zone, advised customers to move workloads to the other us-east-1 zones, and admitted "Customers may experience longer than usual provisioning times." When a zone dies, every correctly built multi-AZ architecture in the region starts replacing capacity in the surviving zones at the same time; the surviving zones are a shared pool and mass failover is a thundering herd against it.

Coinbase's own words

Two sentences carry the architecture story. First: "Our matching engine was pinned to a single building." Second: the Coinbase Exchange matching engine "runs as a Raft-based replicated cluster inside an AWS Cluster Placement Group". And the sentence that turns a zone event into an eight-hour outage: "There was no automated cross-zone failover."

A cluster placement group is single-AZ by construction: its point is to pack instances onto physically close hardware for the lowest node-to-node latency EC2 can offer, and you cannot have that and zone separation at once, because zone separation is distance and distance is latency. For a matching engine it is the correct choice, and Coinbase says so: "We make this choice deliberately." The problem is what lived inside it: five Raft nodes sharing a building share a failure domain, so the protocol can no longer survive the minority loss it exists to survive. When AWS terminated EC2 instances inside the placement group at 9:29 PM ET, three of five matching-engine nodes went down together and quorum was gone. Recovery required an emergency code change, shipped during the incident, to remove a startup assumption about all five cluster nodes being reachable. Cancel-only mode at 2:25 AM ET, full trading at 3:49 AM ET, retail fully recovered by 9:53 AM ET, event-streaming backlog cleared by 2:00 PM ET.

Multi-AZ is something you buy and something you build

The recurring shapes of the gap: the single-subnet Auto Scaling group that replaces failed instances into the same zone forever; RDS without Multi-AZ, where zone loss is a restore measured in hours, not a failover; the quorum (3, 5, or 7 nodes of Raft, etcd, or ZooKeeper) inside a cluster placement group; and the cross-AZ NAT dependency, a private subnet in zone B whose default route points at a NAT gateway in zone A. None are exotic, and all pass load tests and instance-level game days, because each fails only when a whole zone does.

The lab: a single-AZ concentration detector

Lab 01-single-az-detector in the companion repo is a single Python file that scans the current account and region for exactly these patterns. Strictly read-only: every API call is a Describe*. Seven checks: SINGLE_AZ_ASG, CLUSTER_PLACEMENT_GROUP, QUORUM_IN_ONE_AZ, RDS_SINGLE_AZ, RDS_CLUSTER_ONE_AZ, ELASTICACHE_SINGLE_AZ, CROSS_AZ_NAT. QUORUM_IN_ONE_AZ is the CRITICAL one: a cluster placement group holding exactly 3, 5, or 7 running instances, the shape of a consensus quorum, the exact configuration that took Coinbase down. The scan exits 0 whenever it completes, findings or not; a check denied by IAM is skipped with a stderr warning and the rest continues; exit 2 is reserved for a scan that cannot run at all. Every finding reports the zone as both name and ID, for example us-east-1a (use1-az4): the name is an alias randomized per account, the ID is the physical zone, and without the ID you cannot tell whether the zone in a Health Dashboard notice was yours.

Failure modes and what to watch

The trade-off is legitimate: single-AZ placement buys latency, cross-AZ transfer costs money, synchronous replication costs write latency. Coinbase engineered into the placement group deliberately; the failure was that the consequence, no automated path out of the zone, was discovered mid-incident instead of decided in a design review. Watch quorum arithmetic, not just location (two of three etcd nodes in one zone is the same defect); ASGs whose several subnets all resolve to one zone; caches without replicas, where zone loss lands a miss storm on the database; and provisioning capacity during a real zone event, which no detector can see because it is a property of everyone else's architecture. The zone boundary is real, and it held. What you build on top of it is yours.

Read this next: Part 2 of this series, Two Zones, One Cause. Multi-Tenant LLM Apps: Isolating Customers on a Shared Model at ercan.ai (https://ercan.ai/multi-tenant-llm-apps-isolation/). Companion repo: https://github.com/flightlesstux/nothing-fails-alone

---

# Two Zones, Eighteen Hours, One Cause

URL: https://ercan.cloud/nothing-fails-alone-part-2-two-zones-one-cause/
Date: 2026-06-17T09:00:00+02:00

On March 1, an availability zone in AWS's me-central-1 region, mec1-az2, was hit by what AWS first described as "objects that struck the data center, creating sparks and fire". About eighteen hours later, a second zone, mec1-az3, lost power too. The third, mec1-az1, stayed up throughout. With two of three zones impaired, S3 and DynamoDB began failing region-wide, and AWS's advice to customers, in its own words on the Health Dashboard, was to "enact their disaster recovery plans and recover from remote backups into alternate AWS Regions, ideally in Europe". Availability zone independence is engineered against power, cooling, and network faults; in March it met a cause it was never engineered against.

Part 2 of Nothing Fails Alone. Part 1 argued the zone boundary held in May's us-east-1 incident, and it did, for the failure modes the boundary is built for. The March event asks a different question: what happens when one cause reaches more than one zone. It read as a wartime special case until May made the general shape visible.

What actually happened in me-central-1

The Health Dashboard event opened at 4:51 AM PST on March 1 as a "localized power issue" in mec1-az2. At 9:41 AM PST AWS explained: "At around 4:30 AM PST, one of our Availability Zones (mec1-az2) was impacted by objects that struck the data center, creating sparks and fire. The fire department shut off power to the facility and generators as they worked to put out the fire." The next day AWS named the cause: drone strikes, part of the conflict in the Middle East. "In the UAE, two of our facilities were directly struck", causing "structural damage" and disrupted power delivery. Through the first day it was a textbook single-zone event; AWS even noted that customers running redundantly across AZs were not impacted. Then, at 10:46 PM PST, roughly eighteen hours after the strike: "a localized power issue has affected another Availability Zone in the ME-CENTRAL-1 Region (mec1-az3) ... At this point it is not possible to launch new instances in the region ... Other AWS Services, such as DynamoDB and S3 are also experiencing significant error rates and latencies."

Eighteen hours is not simultaneous, and that is the point

The zones did not fall together; they fell about eighteen hours apart, to the same cause. Zone independence is a probability claim about internal faults: separate power, cooling, flood plains, network paths. On its own terms it worked; neither zone took the other down. What failed was the independence assumption itself. A cause that originates outside the facilities, a storm, a grid collapse, a wildfire, an armed conflict, is sampled once and applied to every zone in its reach. Correlated failure does not mean two zones die in the same instant; it means one cause reaches both on its own schedule. The eighteen-hour gap is what correlation looks like on a dashboard: the same hand knocking twice.

When the region itself is the failure domain

At 2:53 AM PST on March 2, AWS explained what two lost zones do to S3: "Amazon S3 is a regional service and designed to withstand the total loss of a single Availability Zone ... As the second AZ became impaired, S3 error rates increased. With two Availability Zones significantly impacted, customers are seeing high failure rates for data ingest and egress." S3 and DynamoDB are the services you do not architect for zone failure because AWS does it for you, with a stated tolerance of one zone. The second zone exceeded it, and the regional substrate every multi-AZ architecture leaned on refused reads and writes region-wide. mec1-az1 was healthy and it did not matter. The regional services are built on the same three zones you are, and their tolerance for correlated zone loss is a number AWS picked, not a number you picked.

"Ideally in Europe"

By 6:22 AM PST on March 2: "We recommend customers enact their disaster recovery plans and recover from remote backups into alternate AWS Regions, ideally in Europe." The provider whose region it is told customers to leave it: that is the ceiling of in-region redundancy, stated by the vendor. "Ideally in Europe" also rules out the reflexive nearby choice, me-south-1 in Bahrain, where a facility had been affected by a nearby strike in the same campaign: the correlated cause had a geography, and a useful recovery region is outside that geography. And "recover from remote backups" presumes the backups exist, created before March 1, because S3 ingest and egress were failing region-wide during the event. Copying data out of a region is a peacetime activity.

Several months

The event's last update, April 30: the region "has suffered damage as a result of the conflict in the Middle East and is currently unable to reliably support customer applications", customers should migrate all accessible resources to other Regions, billing operations are suspended, and recovery "is expected to take several months". An RTO written on the assumption of in-region recovery is not stressed by that, it is deleted by it. The real recovery time becomes a cold cross-region rebuild: IAM and networking from code, data from replicas made beforehand, DNS cutover, capacity in a destination region absorbing everyone's exodus at once.

The lab: which physical zones are you actually in?

The incident reports said mec1-az2 and mec1-az3. A zone name like me-central-1a is an account-scoped alias: AWS assigns letters to physical zones independently per account, so "we are spread across a, b and c" says nothing about physical placement. Lab 02-az-id-truth is a single read-only script whose only API calls are DescribeAvailabilityZones and GetCallerIdentity; it prints the name-to-ID mapping for the current account and region. In the account it was run against, use1-az4, the zone in May's report, is us-east-1c, not us-east-1a. A --compare flag prints two accounts' mappings side by side with a per-name verdict; the same name resolving to different physical zones across accounts is the normal case. Runbooks, placement decisions, and incident lookups belong in zone IDs. Exit code 0 whenever the run completes; 2 only when the run cannot happen at all.

Failure modes and what to watch

A customer cannot architect me-central-1 out of a war; what you control is whether the correlated cause finds you with your data already elsewhere. Enumerate the causes that reach all your zones at once (shared metro, grid, weather, jurisdiction) and treat that as your real region-loss probability. Hold cross-region backups in a region outside the correlated geography and test the restore. Write RTOs twice, once for in-region recovery and once assuming the region is gone for a quarter. Keep an inventory of physical zone IDs per account, because the time to learn your mapping is not during the event. Zones can be perfectly isolated from each other and still not be isolated from the world.

Read this next: Part 1 of this series, AZ Isolation Held. Your Architecture Didn't. (https://ercan.cloud/nothing-fails-alone-part-1-az-isolation-held/). Part 3 of this series, The Fix Shares Fate. Cross-Region Inference: Cheap Resilience or Residency Trap? at ercan.ai (https://ercan.ai/cross-region-inference-residency/). Companion repo: https://github.com/flightlesstux/nothing-fails-alone

---

# The Fix Shares Fate With the Outage

URL: https://ercan.cloud/nothing-fails-alone-part-3-fix-shares-fate/
Date: 2026-07-01T09:00:00+02:00

Every recovery plan carries a second assumption, quieter than the first. The first is that some of the infrastructure survives the event; Parts 1 and 2 tested that one. The second is that when the event comes, you can still deploy the fix: the API answers, the console loads, your pipeline can assume a role. On June 13th, 2023, a latent defect in the subsystem that manages Lambda's compute capacity degraded function invocations in us-east-1, and for part of the next four hours the AWS Management Console in that region served error pages while STS threw elevated error rates. Nothing in most customers' infrastructure broke that day; what broke was the layer they would have used to fix it. When the control plane is inside the blast radius, the remediation path is inside the failure domain.

Part 3 of Nothing Fails Alone. Part 1 promised this part would be about what your recovery plan shares with everyone else's recovery plan. The answer: it shares the control plane. Your failover calls the same regional APIs as every other failover in the region, operated by the provider, degraded by the same events that make you need it.

June 2023: the fix needed the thing that failed

Per AWS's post-event summary for June 13th, 2023: at 10:01 AM PDT the Lambda Frontend fleet in us-east-1 began scaling for ordinary daily traffic; at 11:49 AM it crossed "a capacity threshold that had previously never been reached within a single cell", which "triggered a latent software defect". The subsystem "responsible for managing the underlying compute capacity" could not provision working execution environments and Lambda invocations in the region started failing. Downstream, the summary lists Amazon STS, the AWS Management Console, Amazon EKS, Amazon Connect, and Amazon EventBridge as degraded "as a result of the degraded Lambda function invocations". STS returned elevated error rates from 11:49 AM to 2:10 PM PDT "with three distinct periods of impact"; the console served "AWS Management Console is currently unavailable" or "504 Time-out" pages from 11:48 AM to 2:02 PM; "Existing IAM sessions were not impacted, but new sign-in federation via SAML was degraded". Full recovery at 3:37 PM PDT. Map that onto a runbook: sign in (degraded), assume a role (degraded, three times), invoke the remediation Lambda (that is the outage). The lesson is structural, not historical.

October 2025: the meltdown, revisited from the control plane

This site covered the October 2025 us-east-1 outage the same day in "When the Cloud Sneezes, the World Catches a Cold" (https://ercan.cloud/when-the-cloud-sneezes-the-world-catches-a-cold-lessons-from-the-us-east-1-meltdown/); this part is the follow-up that post promised, written against AWS's official post-event summary. The trigger was "a latent race condition in the DynamoDB DNS management system that resulted in an incorrect empty DNS record" for dynamodb.us-east-1.amazonaws.com. DynamoDB recovered within hours; the control-plane damage ran longer. EC2: "Between 11:48 PM PDT on October 19 and 1:50 PM PDT on October 20, customers experienced increased EC2 API error rates, latencies, and instance launch failures", fourteen hours during which the standard recovery move of every multi-AZ architecture in the region was impaired. STS threw API errors from 11:51 PM to 9:59 AM; IAM-user console sign-in failed until 1:25 AM; root and federated sign-in failed even outside the region. Scaling out was the fix, and scaling out was down. Everyone's plan converged on the same degraded API at the same time.

February 2026: Azure, same shape

Azure incident FNJ8-VQZ, per Microsoft's post-incident review: "Between 18:03 UTC on 02 February 2026 and approximately 00:30 UTC on 03 February 2026, a platform issue caused some customers to experience degraded performance and control plane failures, for multiple Azure services across multiple regions." A security remediation policy was, through "a data synchronization problem in the targeting logic", applied to storage accounts serving VM extension packages. VMs hit "failures when deploying or scaling", AKS hit "failures in node provisioning", GitHub Actions runners queued and timed out. Running VMs kept running; only the ability to create, scale, or change was gone, across regions at once. Not an AWS quirk: a property of control planes, shared in a way data planes are not, and under maximum load exactly when everyone is recovering.

Static stability, defined properly

A statically stable system keeps meeting its requirements when a dependency fails without having to change anything, because whatever it would have asked the dependency for is already in place. Concretely: pre-provisioned headroom (each zone runs enough that survivors carry full load with no launch, turning zone loss into a load-balancer decision instead of an Auto Scaling event); pre-created rather than created-on-failover (standby DNS records, target attachments, standby instances exist before the incident); pre-baked AMIs that boot to serving without package mirrors or config services; break-glass access that does not traverse the failed region and identity plane, and that is actually exercised. The common thread: move work from during-the-incident to before-the-incident. You cannot make the provider's control plane reliable; you can arrange to not need it for the first hour.

The lab: measure the difference, do not assert it

Lab 03-static-stability is the one lab in the series that is not read-only: Terraform builds a VPC, an ALB, and six t3.micro instances, and costs money until destroyed. AWS's fault-injection documentation states the premise: during an AZ power interruption, "EC2 API calls made by the Auto Scaling control plane to recover lost capacity in the AZ will fail". Path A is an ASG with one instance per zone and a scaling policy: recovery requires ec2:RunInstances. Path B is the same workload pre-provisioned at double capacity, two per zone: either zone alone carries the load. Subnet removal alone is not zone loss (rebalancing launches before terminating, so nothing dips); measure.py also terminates the failed zone's in-service instances via TerminateInstanceInAutoScalingGroup with ShouldDecrementDesiredCapacity=False. Path A is timed from fault to full healthy count, an interval that is pure control plane; path B's minimum healthy count through a settle window is tested against the post-failure threshold, a real dip from four to two. The table it prints reports time to recover, API calls required, and whether the path survives control plane loss; the numbers a reader gets are theirs, and path A's number on a healthy day is its best case. Lab 04 repeats the exercise with insufficient-capacity errors injected.

The honest cost, and where it stops being worth it

Path B runs double steady-state capacity: static stability is insurance paid in idle compute, every hour, against an event measured in hours per year. With two zones the premium is 100 percent; with three zones sized to lose one, 50. Pay it for the serving path whose downtime is measured in revenue or safety, for small fleets, and for anything whose recovery would otherwise depend on a mid-incident control plane. Stop paying where the math stops: batch should queue through zone loss, dev and staging do not need it, and very large fleets should split the tier. Static stability would not have saved Coinbase (their failure was their own data plane) and does not answer region loss (headroom in a doomed region is doomed headroom). It protects the first hours, betting the control plane returns before spare capacity stops being enough.

Read this next: Part 2 of this series, Two Zones, Eighteen Hours, One Cause (https://ercan.cloud/nothing-fails-alone-part-2-two-zones-one-cause/). Part 4 of this series, Testing the Assumption. Agents on Call, Part 2. The Foundation: Terraform Before Tokens at ercan.ai (https://ercan.ai/agents-on-call-part-2-terraform-before-tokens/). Companion repo: https://github.com/flightlesstux/nothing-fails-alone

---

# Testing the Assumption Nobody Tests

URL: https://ercan.cloud/nothing-fails-alone-part-4-testing-the-assumption/
Date: 2026-07-15T09:00:00+02:00

Both load-bearing assumptions in this series are testable today, with a managed service, in an afternoon. AWS Fault Injection Service ships a scenario named "AZ Availability: Power Interruption" that stops every tagged instance in one availability zone, feeds InsufficientInstanceCapacity errors to the Auto Scaling groups trying to replace them, pauses EBS volume IO, and cuts subnet traffic, then lifts the faults and restarts what it stopped. The control-plane assumption from Part 3 has no off-the-shelf fault, but an honest approximation fits in one JSON file and one IAM deny. Almost nobody runs either, and the reason is not tooling; the reason is that a test can fail and an assumption cannot. The series' spine in two sentences: the availability you buy ends at the boundary of what the provider promised, and everything above that line is architecture you own. What you own and never exercise is not architecture, it is an assumption with your name on it.

The AZ drill comes off the shelf

The scenario's published JSON carries seven actions: tagged instances in the zone stopped and restarted after the duration, InsufficientInstanceCapacity injected into tagged ASGs' launch requests and into capacity calls of named IAM roles (per the scenario page, "EC2 API calls made by the Auto Scaling control plane to recover lost capacity in the AZ will fail"), EBS volume IO paused, subnet traffic denied via a cloned network ACL, RDS clusters with a writer in the zone failed over, ElastiCache replication groups zone-power interrupted. Precision about what it is not: instances are stopped through the StopInstances API, an orderly shutdown, not a power cut, so crash consistency goes untested; Fargate tasks are not covered; RDS Multi-AZ with two readable standbys is not supported. One documentation drift: the page describes a recovery action, aws:arc:start-zonal-autoshift, that its own JSON snapshot does not contain. The companion lab, 04-game-day, breaks the series' read-only rule for the second and last time: --inject starts real experiments, so it belongs in a sandbox pointed at Part 3's lab stack.

Names, IDs, and a stop condition shipped empty

Incident reports speak zone IDs (use1-az4, mec1-az2); your account speaks names, aliases AWS randomizes per account; and the tool mostly demands the name. Instance and subnet targets filter on the account-local zone name, and the EBS, RDS, and ElastiCache target parameters take the name too. Exactly two actions accept the physical ID: aws:ec2:api-insufficient-instance-capacity-error and aws:ec2:asg-insufficient-instance-capacity-error, whose availabilityZoneIdentifiers parameter is documented to take zone IDs as well as names; even there the published scenario fills the field with a name. The lab's run.py accepts only --az-id, resolves the account-local name via DescribeAvailabilityZones, prints both, and fills each placeholder with the form its consumer requires. The published scenario's stop-condition block ships with an empty value: the shape of a guardrail with nothing in it, and the limitations section says stop conditions are yours to add. run.py exits before creating anything if --alarm-arn is missing, because an experiment without a meaningful stop condition is not a test, it is a bet.

What a pass looks like, measured

Run against Part 3's two-path stack, failing one of its two zones. The runner polls the experiment to a terminal state, logging action status and sampling healthy-target counts. The shape of a pass, not an obtained result: path B never drops below its post-failure requirement of two healthy targets; path A drops to half and stays there for the whole fault duration, because the same experiment that stopped its instances feeds its replacement launches InsufficientInstanceCapacity. Four numbers worth recording: time to recovered capacity; the count of runbook steps needing a control-plane call that would have failed; the gap between Part 3's cooperative recovery number and the game day's uncooperative one, which is the size of the assumption in your DR plan; and the fate of each Part 1 detector finding, since every row it prints is a prediction the game day can settle.

The control-plane drill FIS cannot give you

FIS has no control-plane-loss action; the full actions reference contains nothing that takes a control plane away. The closest are error injections scoped to callers you name, for the EC2 and Kinesis namespaces only. So the lab's second template is an approximation labelled as one, in a top-level approximation key in the file itself: FIS injects InsufficientInstanceCapacity on the workload ASGs' launches and the deploy role's capacity calls for twenty minutes, and a manual, scoped, time-boxed IAM deny on ec2:RunInstances, autoscaling:*, and cloudformation:* covers the runbook's "just push the emergency stack" step. run.py prints the apply and remove commands and never executes them, and the deny never goes on the role you need to remove it. What it does not reproduce: brownout and latency rather than clean errors; ELB, Route 53, IAM, and the console keep working; reads keep succeeding so dashboards look healthier than they would; non-capacity mutations like TerminateInstances keep succeeding, so it denies capacity creation, not the whole control plane; and it is zone-scoped where a real event can be regional. Measure time to detect: how long before a human noticed the fix was not working.

Scoring the series

Established, on the operators' own published words: zone isolation is real against the faults it is engineered for (May stayed inside use1-az4); zone failures are not independent events (one cause reached mec1-az2 and mec1-az3 eighteen hours apart and pushed regional services past their stated one-zone tolerance); control planes fail while data planes keep running, on more than one provider. Still assumption: everything quantitative about you. These labs never ran against a production fleet, the thundering-herd cost of mass failover has never been given a number, and whether your break-glass path works is unknowable from here. Differently next time: run the game day first, since one injection run produces the finding list the detector can only predict, and distrust secondary sources sooner; re-verifying against primary pages caught wrong durations, a timestamp off by half a day, and claims whose apparent sources did not contain them.

Game days fail too

Three ways a game day lies. The test that only proves the test ran: FIS resolves targets by tag, empty targets are skipped, and an experiment can reach completed having stopped nothing; read the per-action table, not the status line. The stop condition that fires too early: you learn only that the alarm fires, and the fix is binding it to the user-facing steady-state metric, not loosening it. The account that is not production: a sandbox pass proves the mechanism, not the outcome, so the honest chain is detector predicts, sandbox game day tests the mechanism, production inherits a labelled extrapolation. Nothing fails alone, and nothing passes alone either: every pass is entangled with the environment that produced it.

Read this next: Part 3 of this series, The Fix Shares Fate With the Outage (https://ercan.cloud/nothing-fails-alone-part-3-fix-shares-fate/). Part 1 of this series, AZ Isolation Held. Your Architecture Didn't. (https://ercan.cloud/nothing-fails-alone-part-1-az-isolation-held/). Evals Before Agents: You Can't Ship What You Can't Score at ercan.ai (https://ercan.ai/evals-before-agents/). Companion repo: https://github.com/flightlesstux/nothing-fails-alone

---

# FinOps: Budget Alerts with Terraform Before Your Next $2.5B AWS Bill

URL: https://ercan.cloud/finops-budget-alerts-with-terraform/
Date: 2026-07-17T18:00:00+02:00

On July 17, AWS Cost Explorer showed customers estimated bills of millions, billions, and in a few screenshots trillions of dollars. The numbers were wrong, nobody was charged, and AWS fixed the data within a day. The detail worth keeping: the people who noticed first were the ones with billing alerts configured. This post is the 101 on setting those up with Terraform's aws_budgets_budget, from a plain monthly cost alert to usage budgets and Savings Plan utilization tracking.

What happened

At 1:33 AM Pacific, AWS posted to the Health Dashboard that Cost Explorer was "reflecting inaccurate estimated billing data." The root cause, in AWS's words, was "an issue with unit pricing within the estimated billing computation subsystem." The numbers were estimates, not actual usage or charges; nobody was charged; AWS backfilled the data and the numbers were back to normal by July 18. Many people learned about the bug from a billing threshold exceeded email: accounts with alerts configured spotted the anomaly within minutes, accounts without them found out from social media or not at all. This one was a false alarm, but a forgotten NAT Gateway, a Lambda stuck in a retry loop, or a leaked access key produces a bill that is very much real.

Why IaC and not the console

An alert created in the console lives in one account and in one person's memory. With Terraform it lives in version control, a new account gets the same module, and a threshold change is a pull request that goes through review. The resource that does the work is aws_budgets_budget, wrapping AWS Budgets.

The six examples

1. A monthly cost budget with an email alert: budget_type COST, limit_amount 100 USD, time_unit MONTHLY, with a notification block (GREATER_THAN, threshold 80, PERCENTAGE, ACTUAL, subscriber_email_addresses).

2. Forecast-based early warning: notification_type FORECASTED warns before the money is spent when the end-of-month forecast will blow the budget. Pair a FORECASTED notification at threshold 100 with an ACTUAL notification at 90 to cover both directions.

3. Slack or PagerDuty via SNS: point subscriber_sns_topic_arns at an aws_sns_topic and wire the topic to a Slack webhook or PagerDuty. The failure mode: the topic policy must grant SNS:Publish to the budgets.amazonaws.com service principal, otherwise Budgets cannot deliver and alerts disappear silently with no error on the budget side.

4. A per-service budget: add a cost_filter with name Service and values like "Amazon Elastic Compute Cloud - Compute" to watch a single service.

5. A usage budget, 3 GB of S3: FinOps is not only dollars; with budget_type USAGE the budget is defined in usage units (limit_amount 3, limit_unit GB). Useful for staying inside the free tier and catching a growing bucket before it becomes a line item.

6. A Savings Plan utilization budget: budget_type SAVINGS_PLANS_UTILIZATION with limit_unit PERCENTAGE and a cost_types block (include_subscription true, everything else false). The operator inverts: comparison_operator LESS_THAN with threshold 90 fires when utilization drops below 90%, catching a prepaid commitment burning money while workloads run on-demand. The same pattern works for Reserved Instances with RI_UTILIZATION.

Wrapping up

July 17's trillion-dollar screenshots were a display bug; the lesson stands anyway. An anomaly in your bill should reach you through your own alerts, not through your timeline. Define a monthly budget, pair FORECASTED and ACTUAL, route alerts through SNS to a channel the team watches, and complete the picture with usage budgets and Savings Plan utilization tracking.

Read this next: When the Cloud Sneezes, the World Catches a Cold - Lessons from the us-east-1 Meltdown (https://ercan.cloud/when-the-cloud-sneezes-the-world-catches-a-cold-lessons-from-the-us-east-1-meltdown/). Connect your AWS to GCP with Terraform via IPSec Site-to-Site VPN (https://ercan.cloud/connect-your-aws-to-gcp-with-terraform-via-ipsec-site-to-site-vpn/). Agents on Call, Part 2. The Foundation: Terraform Before Tokens at ercan.ai (https://ercan.ai/agents-on-call-part-2-terraform-before-tokens/).

---

# Agent Toolkit for AWS: The Docs Have a New Reader

URL: https://ercan.cloud/agent-toolkit-for-aws-the-docs-have-a-new-reader/
Date: 2026-07-20T09:30:00+02:00

AWS added a new card to the console: Agent Toolkit for AWS. Free, generally available, works with Claude Code, Codex, Cursor, and Kiro. One setup prompt gives a coding agent an MCP server for AWS APIs, curated skills for common tasks, and IAM guardrails. The toolkit is useful; what it admits is bigger: the primary reader of AWS documentation is no longer a person.

What ships in the box

Three independent layers. The AWS MCP Server: a hosted endpoint (aws-mcp.us-east-1.api.aws) reached through mcp-proxy-for-aws, giving the agent AWS API access, sandboxed script execution, and real-time documentation search; credentials are only needed for API calls, documentation search works without them. Plugins: aws-core covers service selection, CDK and CloudFormation, serverless, containers, storage, observability, and billing; aws-agents adds Bedrock and AgentCore skills; aws-data-analytics and aws-agents-for-devsecops cover their domains; on Claude Code they install from the official plugin marketplace. Skills: curated task instructions installed with npx skills add aws/agent-toolkit-for-aws/skills, loaded on demand. Authentication runs through the browser-based aws login flow, verified with aws sts get-caller-identity; aws configure agent-toolkit wires the same through the AWS CLI.

The setup is a prompt, not a script

The console hands you one sentence to paste into your agent, pointing at a raw Markdown file on GitHub. The agent reads it, detects the OS, installs the AWS CLI if missing, runs aws login, wires up the MCP server, and installs the skills. The document is structured like a runbook for a machine: numbered steps, error handling tables, verification commands after every stage. Install instructions used to be prose for humans with code blocks to copy; this is code for agents with prose the human skims once.

Nobody reads the docs anymore, except the agents

For years the author's weekend reading was AWS documentation: service quotas, API edge cases, changelogs. That mental index made engineers fast because knowing where the answer lives was most of the job. The index is now a commodity: an agent with documentation search reads the current page in seconds. What does not become a commodity is judgment: which service fits, the blast radius of a change, whether the agent's plan survives an account's realities. An agent with working credentials is a principal like any other and gets least privilege like any other. Pin the proxy version (mcp-proxy-for-aws@1.6.3 style) against supply chain risk, scope the agent's role read-only by default, and gate every mutating path separately.

Wrapping up

The agentic era of AWS development arrived as a small card in the console offering a setup prompt. AWS now assumes the first collaborator on any project is an agent and ships documentation, procedures, and permission boundaries built for that reader. The interesting work, deciding what to build and what an agent may touch, was always the part that could not be indexed.

Read this next: IaC First: Why We Never Touch the AWS Console in Production (https://ercan.cloud/iac-first-why-we-never-touch-the-aws-console-in-production/). Agents on Call, Part 4. Tools and the Gateway: MCP, Allowlists, Read-Only Default at ercan.ai (https://ercan.ai/agents-on-call-part-4-gateway-read-only-default/).

# Agent Auth on AWS: OIDC In, Access Keys Out

URL: https://ercan.cloud/agent-auth-on-aws-oidc-in-access-keys-out/
Date: 2026-07-21T01:00:00+02:00

An agent running on or against AWS should never hold a long-lived credential. Not an IAM user, not an access key in a secrets manager, not a shared "agents" service account. Every serious authentication path AWS offers, instance profiles, IRSA, EKS Pod Identity, OIDC federation, IAM Roles Anywhere, AgentCore Identity, exists to do the same thing: exchange a verifiable proof of identity for temporary credentials that expire on their own. This post is the AWS layer under the MCP protocol story (SEP-1046 at ercan.ai): a short-lived signed token in, short-lived credentials out.

One pattern under everything: OIDC federation

Strip the product names away and AWS agent auth is one mechanism. An identity provider signs a JWT about a workload. You register that provider's issuer URL in IAM as an OIDC identity provider; IAM fetches the public keys from the provider's jwks_uri via the /.well-known/openid-configuration discovery document. The workload presents its token to sts:AssumeRoleWithWebIdentity and gets role credentials that expire within the hour. The role's trust policy pins who gets in with conditions on the token's aud and sub claims. The version every platform team knows is GitHub Actions: the runner's OIDC token encodes repo and branch in its sub claim, the trust policy matches it, and no access key ever lives in repository secrets. That is the template for every unattended workload, agents included.

Where the agent runs decides the mechanism

On EC2 or ECS: instance profile or task role; the SDK credential chain picks it up. On EKS: IRSA (per-cluster OIDC issuer registered in IAM, ServiceAccount annotation) or the newer EKS Pod Identity (no per-cluster OIDC provider, node agent add-on, trust policy targets pods.eks.amazonaws.com once); for a fleet of clusters Pod Identity removes real toil. Outside AWS with an IdP: plain OIDC federation. Outside AWS without an IdP: IAM Roles Anywhere, X.509 certificates from a CA registered as a trust anchor, with the honest caveat that you inherit certificate lifecycle management.

AgentCore Identity: the agent-shaped layer on top

IAM answers how a workload calls AWS. Agents add two more problems: who may call the agent, and how the agent reaches non-AWS services on a user's behalf. Amazon Bedrock AgentCore Identity covers that slice: per-agent workload identities, inbound auth via SigV4 or a JWT authorizer (discovery URL plus allowed audiences, Cognito, Okta, or Entra ID), and outbound auth through an OAuth token vault holding two-legged and three-legged grants with consent stored server side. Through AgentCore Runtime or Gateway it costs nothing extra; standalone use bills per successful token or API key request. The protocol layer keeps moving too: the MCP spec repo's auth queue holds securitySchemes in tool metadata for mixed-auth servers (SEP-1488), standardized tool errors that trigger an OAuth flow mid-session (SEP-1489), and declared authentication methods for remote servers (SEP-2742).

Attribution is the part audits actually ask about

Temporary credentials solve theft; they do not answer which agent did what. One role per agent so CloudTrail names the workload. sts:SourceIdentity set at assumption time so the triggering human or system survives into downstream logs across role chaining. Session policies to scope a broad role down to the task at hand. Keep standing permissions read-only and gate every mutating action.

What to do on Monday: treat every IAM user with active access keys belonging to an automation as a finding. Map each agent to its runtime and adopt the matching mechanism, in order of preference: instance/task roles, IRSA or Pod Identity, OIDC federation, Roles Anywhere. Pin every OIDC trust policy on aud and the narrowest sub pattern you can write. Set sts:SourceIdentity in every agent's assume-role call. On AgentCore, front agents with the JWT authorizer and move third-party OAuth grants into the token vault. Agent identity is converging on signed short-lived assertions, audience binding, and per-workload principals; teams still copying access keys into agent configs are running 2015 security posture against 2026 workloads.

Read this next: Agent Toolkit for AWS: The Docs Have a New Reader (https://ercan.cloud/agent-toolkit-for-aws-the-docs-have-a-new-reader/). Enterprise MCP Auth: Agents Finally Get Service Accounts at ercan.ai (https://ercan.ai/enterprise-mcp-auth-agents-service-accounts/).