Tithe.ly Engineering

Rails Dependency Licenses

by Shawn Pyle and Dave Orweller on June 2, 2021

We recently had to evaluate all of our repositories for dependency licenses that would potentially cause problems, e.g. mimemagic breaking Rails. It’s better to be proactive about these things, so we decided to do this during a recent coding cool down.

Tools

We had a hard time finding a library that would give us everything we wanted so we created a script to reuse in every repository. It uses the licensed gem underneath with an undocumented JSON option.

We created a custom thor command to handle exporting all licenses to a CSV file and cleaning up any temporary files. Thor is a fantastic library for creating utility Command Line Interface (CLI) scripts. We use it to interact with docker containers, perform deploys, and now checking licenses.

Requirements

The following script is developed for MacOS (i.e. brew and open commands are used) but you could probably adapt to your system pretty readily.

Thor Command

Here is what you came for…

# frozen_string_literal: true

# Copyright 2021 YourGiving.co (DBA Tithe.ly)
#
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

require "json"
require "csv"

module Integrations
  module CLI
    module Command
      class License < Base
        desc "csv", "Generate a CSV file for all dependencies and licenses"
        def csv
          return unless install_dependencies
          install_gem
          create_config
          cache_dependencies
          export_csv
          puts "Finished exporting license metadata to #{csv_path}"
          `open #{csv_path}`
        end

        desc "clean", "Remove license cache and config"
        def clean
          [config_path, cache_path, csv_path].each do |path|
            puts "Removing #{path} ..."
            FileUtils.rm_rf(path) if File.exist?(path)
          end
        end

        private

        def git_root
          @git_root ||= `git rev-parse --show-toplevel`.strip
        end

        def app_name
          @app_name ||= File.basename(git_root)
        end

        def csv_path
          @csv_path ||= File.join(Dir.getwd, "#{app_name}_licenses.csv")
        end

        def install_dependencies
          `which brew`
          has_brew = $?.success?

          %w(cmake pkg-config).each do |library|
            `which #{library}`
            next if $?.success?

            if has_brew
              puts "Installing #{library} with brew..."
              `brew install #{library}`
              unless $?.success?
                puts "Failed to install #{library}"
                return false
              end
            else
              puts "Please install cmake"
              return false
            end
          end

          true
        end

        def install_gem
          if `gem list -i licensed`.strip == "false"
            puts "Installing licensed gem..."
            `gem install licensed`
          end
        end

        def cache_path
          @cache_path ||= File.expand_path(File.join(git_root, '.licenses'))
        end

        def cache_dependencies
          unless Dir.exist?(cache_path)
            puts "Caching licenses..."
            `licensed cache`
          end
        end

        def config_path
          @config ||= File.join(Dir.getwd, ".licensed.yml")
        end

        def create_config
          unless File.exist?(config_path)
            puts "Creating config file #{config_path} ..."
            File.open(config_path, "w") { |f| f.write "name: '#{app_name}'\nsources:\n  bundler: true\n  yarn: true" }
          end
        end

        def json
          @json ||= JSON.parse(`licensed status --format=json`)
        end

        def export_csv
          puts "Exporting licenses to #{csv_path} ..."
          CSV.open(csv_path, "wb") do |out|
            out << ["App", "Source", "Library Name", "Library Version", "License"]
            json["apps"].each do |a|
              a["sources"].each do |s|
                next unless s["dependencies"]

                s["dependencies"].each do |d|
                  app = d["name"].split(".")[0].downcase.gsub(" ", "_")
                  source = d["name"].split(".")[1]
                  library_name = d["name"].split(".")[2].gsub(/-\d+/, "")
                  library_version = d["name"][/-\d+\.\d+\.\d+/].gsub("-", "") if d["name"][/-\d+\.\d+\.\d+/]
                  library_license = d.fetch("errors", []).join(" ").split(": ")[1]
                  errors = d.fetch("errors", []).join(" | ")
                  out << [app, source, library_name, library_version, library_license]
                end
              end
            end
          end
        end
      end
    end
  end
end

We include it in the main CLI script with something like this

require "thor"
require_relative "./cli/command/license"

class Main < Thor
    desc "license <subcommand>", "Dependency license details"
    subcommand "license", Command::License

    # other subcommands #
end

Conclusion

Having a reproducible script is a windfall when you have 10+ repositories to maintain and want to run this type of action regularly.