gnarlyware
https://gnarlyware.com/
Recent content on gnarlywareHugo -- gohugo.iofredrik.pj.nilsson@gmail.com (Fredrik Nilsson)fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)Wed, 01 Sep 2021 00:00:00 +0000Fooled by Average Customer Revenue
https://gnarlyware.com/blog/fooled-by-average-customer-revenue/
Wed, 01 Sep 2021 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/fooled-by-average-customer-revenue/Let’s say you are running a business. You have just on-boarded an (expensive) new agency for acquiring customers. After a month, you want to decide if you should continue using them. You do this by comparing the average revenue of customers brought in by them agency, to average revenue of customers from your existing channels. How many customers from that agency do you need, to make an informed decision?QUALIFY clause is now available in BigQuery
https://gnarlyware.com/blog/qualify-clause-is-now-available-in-bigquery/
Sat, 28 Aug 2021 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/qualify-clause-is-now-available-in-bigquery/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p>When browsing the <a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#qualify_clause">BigQuery docs</a>, I discovered that BigQuery now supports the <code>QUALIFY</code>-clause. At time of writing, it’s in <a href="https://cloud.google.com/products#product-launch-stages">preview</a>, but you can already try it out.</p>
<p>The <code>QUALIFY</code>-clause lets you filter on the output of <a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/analytic-function-concepts">analytic functions</a>. This allows me to scratch an itch I’ve had for a while:</p>
<div id="for-each-distinct-value-in-column-x-find-the-row-with-max-value-in-column-y" class="section level2">
<h2>For each distinct value in column X, find the row with max value in column Y</h2>
<p>A common pattern is: “For each distinct value in column X, find the row where column Y takes it’s maximum value”. You can accomplish this by using <code>ROW_NUMBER() OVER (PARTITION BY X ORDER BY Y DESC) AS rank</code>.</p>
<p>An example of this would be to <strong>find the date at which each of a set of countries reported the largest number of new confirmed covid-19 cases</strong>. We’ll be using the
<code>bigquery-public-data.covid19_open_data</code> dataset.</p>
About
https://gnarlyware.com/about/
Tue, 24 Aug 2021 21:48:51 +0100fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/about/I’m sharing share short snippets and thoughts on computers, probability, data science and machine learning. Texts will usually be related to current projects, when I believe my summaries will be helpful to others.
When I find the time, I hope to flesh this site out a bit, showcasing a bit of my work and sharing some useful methods for real world data science.Lenovo Yoga Slim 7 Pro running Fedora 34 (with WiFi)
https://gnarlyware.com/blog/lenovo-yoga-slim-7-pro-running-fedora-34/
Tue, 24 Aug 2021 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/lenovo-yoga-slim-7-pro-running-fedora-34/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p>I’ve been on the hunt for a $1000 Linux UltraBook for personal use. I finally decided on a Lenovo Yoga Slim 7 Pro, with a Ryzen 7 5800 CPU and 16 GB RAM. I’ll leave the in-depth hardware review for others, but I’ll offer a nano-review: I’m happy with both build quality and specs. Best value for money I’ve seen in a while!</p>
<p>I’m less happy about the Realtek 8852 WiFi module not having built-in support in the 5.13 Linux kernel. Everything else worked out of the box with a fresh installation of <a href="https://fedoraproject.org">Fedora 34</a> (my distro of choice).</p>
<p>Drivers for the Realtek 8852 module can be downloaded <a href="https://github.com/lwfinger/rtw89">here</a>, but the included Makefile doesn’t work with Fedora if you’re using secure boot. I’ll provide some quick notes below on how to install the drivers on Fedora 34.</p>
<p><strong><em>You’ll still need to install a few packages, and get code on your computer. Try using your phone as a USB tethered modem. If you need to purchase additional hardware (USB network interface), consider just replacing the WiFi module at probably the same cost.</em></strong></p>
Mutual Information – Example with categorical variables
https://gnarlyware.com/blog/mutual-information-exaxmple-with-categorical-variables/
Mon, 19 Apr 2021 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/mutual-information-exaxmple-with-categorical-variables/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p><a href="https://en.wikipedia.org/wiki/Mutual_information">Mutual information</a> and its cousin, the <a href="https://en.wikipedia.org/wiki/Uncertainty_coefficient">Uncertainty coefficient</a> (Theil’s U) are useful tools from Information Theory for discovering dependencies between variables that are not necessary described by a linear relationship.</p>
<p>Plenty of good material already exists on the subject: see Section 1.6 in <a href="https://www.microsoft.com/en-us/research/people/cmbishop/prml-book/">“Pattern Recognition and Machine Learning” by Bishop, freely available as PDF online</a>. What I hope to contribute here are some trivial examples, to help build intuition, based on categorical/multinomial variables.</p>
Advent of Code 2020 - Solutions in racket
https://gnarlyware.com/blog/advent-of-code-2020/
Tue, 01 Dec 2020 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/advent-of-code-2020/ Advent of Code 2020 has finally started! I’ve been looking forward to it, as it will be a good excuse to brush off my absolute favorite general purpose language: Racket.
Follow my solutions on: https://github.com/AvitusA/adventofcode2020
Racket
KL Divergence Online Demo
https://gnarlyware.com/blog/kl-divergence-online-demo/
Thu, 05 Mar 2020 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/kl-divergence-online-demo/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p>To try out Shiny, I created an interactive visualization for Kullback-Leibler divergence (or KL Divergence). Right now, it only supports two univariate Gaussians, which should be sufficient to build some intuition. Proving the set-covering problem is NP-complete (using reduction from the vertex cover problem)
https://gnarlyware.com/blog/proving-the-set-covering-problem-is-np-complete-using-reduction-from-the-vertex-cover-problem/
Tue, 19 Mar 2019 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/proving-the-set-covering-problem-is-np-complete-using-reduction-from-the-vertex-cover-problem/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p>In this post, we will prove that the decision version of the <a href="https://en.wikipedia.org/wiki/Set_cover_problem">set-covering problem</a> is NP-complete, using a reduction from the <a href="https://en.wikipedia.org/wiki/Vertex_cover">vertex covering problem</a> (which is NP-complete). This is Exercise 35.3-2 in <a href="https://mitpress.mit.edu/books/introduction-algorithms-third-edition">CLRS3</a>.</p>
<p>We will follow the template given <a href="https://gnarlyware.com/blog/how-to-show-a-problem-is-np-complete">in an earlier post</a>.</p>
<div id="problem-statement" class="section level2">
<h2>Problem Statement</h2>
<p>We seek to answer whether a set cover <span class="math inline">\(\mathcal{C}\)</span> exists for <span class="math inline">\(X\)</span> (the set containing the elements to be covered), containing a maximum of <span class="math inline">\(k\)</span> sets.</p>
How to show a problem is NP-complete
https://gnarlyware.com/blog/how-to-show-a-problem-is-np-complete/
Mon, 18 Mar 2019 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/how-to-show-a-problem-is-np-complete/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p>This is the first post in a series of posts where I will attempt to give visual, easy to understand, proofs of NP-completeness for a selection of decision problems. I created these while studying them back in college. Hopefully, they’ll be helpful for someone.</p>
<p>In this post, I will give a “template” which can be used (and will be used for the proofs I post).
Proving 0-1 integer programming is NP-complete (using reduction from 3-CNF-SAT)
https://gnarlyware.com/blog/proving-0-1-integer-programming-is-np-complete-using-reduction-from-3-cnf-sat/
Mon, 18 Mar 2019 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/proving-0-1-integer-programming-is-np-complete-using-reduction-from-3-cnf-sat/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p>In this post, we will prove that <a href="https://en.wikipedia.org/wiki/Integer_programming">0-1 integer programming</a> is NP-complete using a reduction from <a href="https://en.wikipedia.org/wiki/Boolean_satisfiability_problem#3-satisfiability">3-CNF-SAT</a> (which is NP-complete).</p>
<p>We will follow the template given <a href="https://gnarlyware.com/blog/how-to-show-a-problem-is-np-complete">in an earlier post</a>.</p>
<div id="problem-statement" class="section level2">
<h2>Problem Statement</h2>
<p>The decision problem for 0-1 integer programming is formulated as follows:</p>
<p>Given an integer <span class="math inline">\(m \times n\)</span> matrix <span class="math inline">\(A\)</span> and an integer <span class="math inline">\(m\)</span>-vector <span class="math inline">\(b\)</span>, determine whether there exists an integer <span class="math inline">\(n\)</span>-vector <span class="math inline">\(x\)</span> with elements in <span class="math inline">\(\{0, 1\}\)</span>, such that:</p>
<p><span class="math display">\[
Ax \le b
\]</span>
Proving set-partition problem is NP-complete (using reduction from subset sum)
https://gnarlyware.com/blog/proving-set-partition-problem-is-np-complete-using-reduction-from-subset-sum/
Mon, 18 Mar 2019 00:00:00 +0000fredrik.pj.nilsson@gmail.com (Fredrik Nilsson)https://gnarlyware.com/blog/proving-set-partition-problem-is-np-complete-using-reduction-from-subset-sum/
<script src="{{< blogdown/postref >}}index_files/header-attrs/header-attrs.js"></script>
<p>In this post, we will prove that the set-partition problem is NP-complete using a reduction from the <a href="https://en.wikipedia.org/wiki/Subset_sum_problem">subset sum problem</a> (which is NP-complete). This is also happens to be Exercise 34.5-5 in <a href="https://mitpress.mit.edu/books/introduction-algorithms-third-edition">CLRS3</a>.</p>
<p>We will follow the template given <a href="https://gnarlyware.com/blog/how-to-show-a-problem-is-np-complete">in an earlier post</a>.</p>
<div id="problem-statement" class="section level2">
<h2>Problem statement</h2>
<p>Given a set <span class="math inline">\(S\)</span> of numbers, determine whether <span class="math inline">\(S\)</span> can be partitioned into <span class="math inline">\(A\)</span> and <span class="math inline">\(\overline{A} = S - A\)</span>, such that:
<span class="math display">\[
\sum_{x \in A} x = \sum_{x \in \overline{A}} x
\]</span>