<?xml version="1.0" encoding="utf-8"?>
    <feed xmlns="http://www.w3.org/2005/Atom">
      <id>https://wolfgirl.dev/blog/</id>
      <title>PolyWolf's Blog - #aws</title>
      <subtitle
        >all posts tagged "aws" on a blog written by PolyWolf</subtitle
      >
      <icon>/apple-touch-icon.png</icon>
      <link href="/tags/aws/rss.xml" rel="self" />
      <link href="https://wolfgirl.dev/blog/" />
      <updated>2025-03-15T14:41:07Z</updated>
      
    <entry>
      <title><![CDATA[I Just Need To Optimize My AWS Egress Patterns]]></title>
      <link rel="alternate" href="https://wolfgirl.dev/blog/2025-03-15-i-just-need-to-optimize-my-aws-egress-patterns/" />
      <id>https://wolfgirl.dev/blog/2025-03-15-i-just-need-to-optimize-my-aws-egress-patterns/</id>
      <published>2025-03-15T14:41:07Z</published>
      <updated>2025-03-15T14:41:07Z</updated>
      <summary><![CDATA[To preface: I've been doing pretty well for myself at minimizing my cloud spend. I just have a static site in S3 behind CloudFront, so th...]]></summary>
      <content type="html"><![CDATA[<p>To preface: I've been doing pretty well for myself at minimizing my cloud spend. I just have a static site in S3 behind CloudFront, so the following table accounts for all of my spend<sup class=footnote-ref><a data-footnote-ref href=#fn-1 id=fnref-1>1</a></sup>:</p><div class=table-wrapper><table><thead><tr><th align=left>Usage type</th><th align=left>Oct 2024</th><th align=left>Nov 2024</th><th align=left>Dec 2024</th><th align=left>Jan 2025</th><th align=left>Feb 2025</th></tr></thead><tbody><tr><td align=left>S3 Requests-Tier1</td><td align=left>$0.03</td><td align=left>$0.02</td><td align=left>$0.02</td><td align=left>$0.07</td><td align=left>$0.10</td></tr><tr><td align=left>S3 TimedStorage-ByteHrs</td><td align=left>$0.05</td><td align=left>$0.05</td><td align=left>$0.05</td><td align=left>$0.04</td><td align=left>$0.03</td></tr><tr><td align=left>S3 Requests-Tier2</td><td align=left>$0.02</td><td align=left>$0.02</td><td align=left>$0.02</td><td align=left>$0.02</td><td align=left>$0.02</td></tr><tr><td align=left><strong>Total costs</strong></td><td align=left><strong>$0.10</strong></td><td align=left><strong>$0.09</strong></td><td align=left><strong>$0.09</strong></td><td align=left><strong>$0.12</strong></td><td align=left><strong>$0.15</strong></td></tr></tbody></table></div><p>So clearly, I don't <em>need</em> to optimize my spend with it in the pennies n all, but there's still a worrying trend, and I like optimizing, so optimize I shall.</p><h2 id=heading-what-happened-in-january>What Happened In January?<a aria-label="Link to heading 'What Happened In January?'" data-heading-content="What Happened In January?" class=anchor href=#what-happened-in-january></a></h2><p>Looking at CloudWatch metrics for my CloudFront distributions, we see that neither the number of requests nor the number of bytes downloaded lines up with the price increase I'm seeing<sup class=footnote-ref><a data-footnote-ref href=#fn-s3 id=fnref-s3>2</a></sup><sup class=footnote-ref><a data-footnote-ref href=#fn-bots id=fnref-bots>3</a></sup>:</p><figure><div class=table-wrapper><table><tbody><tr><td><p><img alt="Requests summed over 7 day periods. Green spikes in October, later November & early December, then tapers off in January and February. Blue is flat for most the rest, with Red and Orange at the very bottom." src=https://static.wolfgirl.dev/polywolf/blog/019599de-a4ef-7665-97df-7b1373ff6cfa/Screenshot%202025-03-15%20095227.png></p></td><td><p><img alt="Bytes downloaded summed over 7 day periods. Red & Orange spike in December and Feburary, with other smaller spikes still high above Green & Blue." src=https://static.wolfgirl.dev/polywolf/blog/019599de-a4ef-7665-97df-7b1373ff6cfa/Screenshot%202025-03-15%20095254.png></p></td></tr></tbody></table></div><figcaption>Green & Blue are the main sites, and Red & Orange are my static files</figcaption></figure><p>Instead, what I think is happening is that I started <a href=https://wolfgirl.dev/blog/2025-01-19-i-have-numbers-now/>analyzing my CloudFront access logs</a>. Whoops!</p><h2 id=heading-whats-wrong-with-the-access-log-egress-pattern>What's Wrong With The Access Log Egress Pattern?<a aria-label="Link to heading 'What's Wrong With The Access Log Egress Pattern?'" data-heading-content="What's Wrong With The Access Log Egress Pattern?" class=anchor href=#whats-wrong-with-the-access-log-egress-pattern></a></h2><p>I have two hypotheses:</p><ol><li>You're not supposed to put the files from <a href=https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html>CloudFront access logs</a> into <code>{yyyy}/{MM}/{dd}</code> folders, because <code>LIST</code> requests are 10x as expensive as <code>GET</code> requests, or</li><li>There isn't a way to use the S3 access logs w/o incurring this cost, because CloudFront just loves to put a bunch of tiny files in ur bucket, and <code>GET</code>ing all those files will incur costs.</li></ol><p>Unfortunately, I can't really think of any way to solve this that doesn't involve services that are more expensive (Firehose, CloudWatch Logs)<sup class=footnote-ref><a data-footnote-ref href=#fn-2 id=fnref-2>4</a></sup> or fetching the files in some other way (AWS Lambda behind API Gateway behind CloudFront) which would still eat most of the same <code>LIST</code>/<code>GET</code> costs.</p><h2 id=heading-lets-do-some-math>Let's Do Some Math<a aria-label="Link to heading 'Let's Do Some Math'" data-heading-content="Let's Do Some Math" class=anchor href=#lets-do-some-math></a></h2><p>I have about 15000 objects in my log buckets, across 60 days. I check it about once a day, so looking at the <a href=https://aws.amazon.com/s3/pricing/>S3 pricing page</a> & accounting for the slightly nested nature of some of the folders, we get:</p><pre class=syntax-highlighting><code>$ python
Python 3.11.11 (main, Dec  3 2024, 17:20:40) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 15000 * 0.0004 / 1000
0.006
>>> 30 * 65 * 0.005 / 1000
0.00975
</code></pre><p>hrm. That's not right. That should only show up as $0.02 on my bill, and clearly it's more than that, so I'm doing something wrong. Maybe it has to make a <code>HEAD</code> request to check if an object's been updated?</p><pre class=syntax-highlighting><code>>>> 30 * 15000 * 0.0004 / 1000
0.18
</code></pre><p>Now that's roughly the same order of magnitude, but now it's <em>too</em> large. Maybe I'm not actually checking my charts every day, but maybe I'm still doing my math wrong... Wait a minute, what about the <code>PUT</code> requests CloudFront makes to upload these objects in the first place?</p><pre class=syntax-highlighting><code>>>> 15000 * 0.005 / 1000
0.07
</code></pre><p>Bingo, we have a winner. Doesn't like up perfectly because I accidentally nuked part of it once, but yeah that'd do it, especially when combined with the uploads I do to my blog normally. Welp, nothing much I can do about that, mystery solved! So long as I <em>actually</em> run dashboard creation just once a day (or even more if I got that <code>LIST</code> calculation wrong), I think the cost will remain manageable (&lt;$0.20); I've paid a lot more for not understanding the Oracle Cloud "Always Free" tier lol.</p><p>Log ingestion is hard!! Still working at it, GoAccess is really nice but still needs a way to get the data. Same with whatever dashboard software I decide to use next<sup class=footnote-ref><a data-footnote-ref href=#fn-observable id=fnref-observable>5</a></sup>.</p><section class=footnotes data-footnotes><ol><li id=fn-1><p>I host pretty much everything else on an Oracle Cloud Always Free VPS, which is quite beefy, at the expense of having to give Larry Ellison my credit card info & having the threat of the VPS being totally deleted for no reason at any point. Still 100% worth it for me, as you can see, with the rest of my spend in the pennies. <a aria-label="Back to reference 1" class=footnote-backref data-footnote-backref data-footnote-backref-idx=1 href=#fnref-1>↩</a></p></li><li id=fn-s3><p>I know data out from CloudFront isn't a perfect analogue for S3 data charges, but it's what I have since turns out S3 doesn't enable access metrics by default. I've turned them on for my log buckets now, hopefully this doesn't start charging me something ridiculous... <a aria-label="Back to reference 2" class=footnote-backref data-footnote-backref data-footnote-backref-idx=2 href=#fnref-s3>↩</a></p></li><li id=fn-bots><p>Every time I look at this chart I'm reminded I should probably put a <code>robots.txt</code> file in my static sites so scrapers don't download a bunch of files... <a aria-label="Back to reference 3" class=footnote-backref data-footnote-backref data-footnote-backref-idx=3 href=#fnref-bots>↩</a></p></li><li id=fn-2><p>At least I'm pretty sure, looking at their pricing pages vs the S3 page and thinking about what my usage pattern looks like. Can't be 100% certain until I actually tried to build things with them, this is a classic AWS-ism :) <a aria-label="Back to reference 4" class=footnote-backref data-footnote-backref data-footnote-backref-idx=4 href=#fnref-2>↩</a></p></li><li id=fn-observable><p><a href=https://observablehq.com/framework/>Observable Framework</a> looks pretty nice... i &lt;3 niche SSGs <a aria-label="Back to reference 5" class=footnote-backref data-footnote-backref data-footnote-backref-idx=5 href=#fnref-observable>↩</a></p></li></ol></section>]]></content>
      <author>
        <name>PolyWolf</name>
        <uri>https://wolf.girl.technology/</uri>
      </author>
    </entry>
  
    <entry>
      <title><![CDATA[I Have "Numbers" Now]]></title>
      <link rel="alternate" href="https://wolfgirl.dev/blog/2025-01-19-i-have-numbers-now/" />
      <id>https://wolfgirl.dev/blog/2025-01-19-i-have-numbers-now/</id>
      <published>2025-01-19T20:59:14Z</published>
      <updated>2025-01-19T20:59:14Z</updated>
      <summary><![CDATA[My blog is hosted with CloudFront pointed at an S3 bucket, automatically updated to whenever I push a commit to the GitHub repository con...]]></summary>
      <content type="html"><![CDATA[<p>My blog is hosted with CloudFront pointed at an S3 bucket, automatically updated to whenever I push a commit to the GitHub repository containing the sources. Pretty simple in the end, but required a lot of reading AWS blog posts to get set up correctly initially. Now that it's no longer on Cohost, I have the opportunity to look at something I haven't been able to before: <strong><em>numbers</em></strong>. Unfortunately this required reading yet more AWS blog posts.!</p><h2 id=heading-where-do-numbers-come-from>Where Do Numbers Come From?<a aria-label="Link to heading 'Where Do Numbers Come From?'" data-heading-content="Where Do Numbers Come From?" class=anchor href=#where-do-numbers-come-from></a></h2><p>From looking at an automatically-created CloudWatch (AWS's logging/metrics collection service) dashboard, I could see that CloudFront (AWS's CDN service) was already collecting... something...</p><p><img alt="CloudWatch dashboard showing total requests and total bytes downloaded from wolfgirl.dev and static.wolfgirl.dev" src=https://static.wolfgirl.dev/polywolf/blog/0194801b-532b-7224-8ae3-2fdeb7168007/Screenshot%202025-01-19%20151439.png></p><p>Great! So my website is getting hits! But on what pages? That was the question I wanted answered, and this dashboard would not or could not answer it.</p><p>Looking further, I found documentation for <a href=https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/standard-logging.html>how to turn on CloudFront access logs</a>, which would contain the information I wanted in the <code>cs-uri-stem</code> field, among others. The data in CloudWatch currently was just "metrics", not "logs", though. If I wanted to put logs in CloudWatch, it would be... some unknown amount amount of money, based on usage. Given an example on the <a href=https://aws.amazon.com/cloudwatch/pricing/>CloudWatch pricing page</a>, I estimated about $1/mo.</p><p>Now that seems pretty reasonable, no? But currently, I pay just $0.10/mo for S3 storage, and nothing for anything else, so logs alone would 10x my costs :(. I am ruthless about making sure I pay as little as possible for the cloud services I use, so this was a non-starter.</p><p>Fortunately, there is another option: CloudFront can put its access logs directly into an S3 bucket, which I could analyze & rotate at my own convenience! Except... how?</p><h2 id=heading-where-do-numbers-go>Where Do Numbers Go?<a aria-label="Link to heading 'Where Do Numbers Go?'" data-heading-content="Where Do Numbers Go?" class=anchor href=#where-do-numbers-go></a></h2><p>Initially, I thought "oh, I can just write a hacky Python script to download & parse them all". But then the thought of writing a hacky Python script seemed really unappealing to me, so I <a href=https://social.treehouse.systems/@PolyWolf/113851932867864276>lazyweb-ed on Mastodon</a> to see if anyone knew of any pre-built solutions to parse CloudFront logs. Fortunately <a href=https://woof.tech/@unlobito/113851963653376831>someone responded</a> (<a href=https://lobi.to/>unlobito</a>) suggesting a program called <a href=https://goaccess.io/>GoAccess</a><sup class=footnote-ref><a data-footnote-ref href=#fn-1 id=fnref-1>1</a></sup> that looked pretty good. Now, all I had to do was write some hacky Python that calls the following shell script!</p><pre class=syntax-highlighting><code class=language-bash><span class=comment>#!/usr/bin/env bash</span>

<span class=comment># Make it so if any individual step fails, the script exits early, among other things</span>
<span class=function>set</span> <span class=constant>-euo</span> pipefail
<span class=comment># Make it so ** acts as a recursive glob (not default on certain bash-es)</span>
<span class=function>shopt</span> <span class=constant>-s</span> globstar

<span class=keyword>if</span> [[ -z <span class=string>"</span><span class=embedded>${</span><span class=property>AWS_BUCKET_NAME</span><span class=embedded>+x}</span><span class=string>"</span> ]]; <span class=keyword>then</span>
  >&2 <span class=function>echo</span> <span class=string>"Must get bucket passed as AWS_BUCKET_NAME"</span>
  <span class=function>exit</span> 1
<span class=keyword>fi</span>

<span class=property>output_folder</span>=<span class=string>"/tmp/</span><span class=embedded>${</span><span class=property>AWS_BUCKET_NAME</span><span class=embedded>}</span><span class=string>"</span>
<span class=function>mkdir</span> <span class=constant>-p</span> <span class=operator>$</span><span class=property>output_folder</span>
<span class=property>output_file</span>=<span class=string>"/tmp/</span><span class=embedded>${</span><span class=property>AWS_BUCKET_NAME</span><span class=embedded>}</span><span class=string>.html"</span>
<span class=function>touch</span> <span class=operator>$</span><span class=property>output_file</span>

<span class=comment># Download all logs from S3. Must also receive environment variables that allow this to work</span>
<span class=function>aws</span> s3 sync <span class=constant>--delete</span> <span class=string>"s3://</span><span class=embedded>${</span><span class=property>AWS_BUCKET_NAME</span><span class=embedded>}</span><span class=string>"</span> <span class=string>"</span><span class=embedded>${</span><span class=property>output_folder</span><span class=embedded>}</span><span class=string>"</span> <span class=number>1</span>>&2

<span class=comment># Generate the report with GoAccess running inside Docker</span>
<span class=function>zcat</span> <span class=string>"</span><span class=embedded>${</span><span class=property>output_folder</span><span class=embedded>}</span><span class=string>/**/*.gz"</span> <span class=operator>|</span> <span class=function>docker</span> run <span class=constant>--rm</span> <span class=constant>-i</span> <span class=constant>-v</span> <span class=string>"</span><span class=embedded>${</span><span class=property>output_file</span><span class=embedded>}</span><span class=string>:/report.html"</span> <span class=constant>-e</span> LANG=<span class=operator>$</span><span class=property>LANG</span> allinurl/goaccess <span class=constant>-a</span> <span class=constant>-o</span> report.html <span class=constant>--log-format</span> CLOUDFRONT - <span class=number>1</span>>&2

<span class=function>echo</span> <span class=string>"</span><span class=embedded>${</span><span class=property>output_file</span><span class=embedded>}</span><span class=string>"</span></code></pre><p>GoAccess is pretty fast, and S3 doesn't take <em>that</em> long to download<sup class=footnote-ref><a data-footnote-ref href=#fn-2 id=fnref-2>2</a></sup>, so I can get updated reports pretty easily. Now, drumroll please...</p><h2 id=heading-whats-my-most-viewed-page>What's My Most Viewed Page?<a aria-label="Link to heading 'What's My Most Viewed Page?'" data-heading-content="What's My Most Viewed Page?" class=anchor href=#whats-my-most-viewed-page></a></h2><p><img alt="A table showing /blog/rss.xml, /, and /cybersec/rss.xml being the most-hit pages" src=https://static.wolfgirl.dev/polywolf/blog/0194801b-532b-7224-8ae3-2fdeb7168007/Screenshot%202025-01-19%20154654.png></p><p>It's RSS.</p><p>Guess I have a few readers huh! Hi 👋😸</p><h2 id=heading-other-neat-insights-i-didnt-expect>Other Neat Insights I Didn't Expect<a aria-label="Link to heading 'Other Neat Insights I Didn't Expect'" data-heading-content="Other Neat Insights I Didn't Expect" class=anchor href=#other-neat-insights-i-didnt-expect></a></h2><ul><li>Firefox is outsize represented (50%) compared to it's overall market share (3%). Y'all are some nerds huh :3</li><li>The "bot problem" (automated vulnerability scanners, other crawlers) isn't <em>quite</em> a big a problem as I thought. RSS traffic is way higher.</li><li>The list of referring sites is pretty interesting; it's mostly feed syndicators, but seeing the range of ones I didn't know about is cool.</li></ul><p>I'll try not to abuse my newfound power <em>too</em> hard. Nice to have it available tho.</p><section class=footnotes data-footnotes><ol><li id=fn-1><p>Somehow written in C, not Go?? lol <a aria-label="Back to reference 1" class=footnote-backref data-footnote-backref data-footnote-backref-idx=1 href=#fnref-1>↩</a></p></li><li id=fn-2><p>Hopefully, I haven't tested it on days/weeks worth of data, only hours. <a aria-label="Back to reference 2" class=footnote-backref data-footnote-backref data-footnote-backref-idx=2 href=#fnref-2>↩</a></p></li></ol></section>]]></content>
      <author>
        <name>PolyWolf</name>
        <uri>https://wolf.girl.technology/</uri>
      </author>
    </entry>
  
    </feed>
  