// thinkbeforecoding

fck: Fake Construction Kit

2016-12-04T10:34-56 / jeremie chassaing

Yeah it's christmas time again, and santa's elves are quite busy.

And when I say busy, I don't mean:

Santa's elves

I mean busy like this:

Santa's elves

So they decided to build some automation productivity tools, and they choose Santa's favorite language to do the job:

F# of course !

F# scripting

No body would seriously use a compiled language for automation tools. Requiring compilation or a CI server for this kind of things usually kills motivation.

Of course it is possible to write bash/batch files but the syntax if fugly once you start to make more advanced tools.

Python, JavaScript, Ruby or PowerShell are cool, but you end up as often with scripted languages with dynamic typing which you'll come to regret when you have to maintain it on the long term.

F# is a staticaly typed language that can be easily scripted. Type inference make it feel like shorter JavaScript but with far higher safety !

Writing F# script is easy and fast. Test it from the command line:

1: 
vim test.fsx

Then write:

1: 
printfn "Merry Christmas !"

press :q to exit

now launch it on linux with:

1: 
fsharpi --exec test.fsx

or on windows:

1: 
fsianycpu --exec test.fsx

Excellent.

The only problem is that typing the fshapi --exec this is a bit tedious.

Bash/Batch to the rescue

We can create a bash/batch script to puth in the path that will launch the script (for linux):

1: 
vim test
1: 
fsharpi --exec test.fsx
1: 
chmod +x test

or one windows

1: 
vim test.cmd
1: 
fsianycpu --exec test.fsx    

Done !

Better, but now we need to write a bash and/or a batch script for each F# script.

fck bash/batch dispatcher FTW !

We create a fck file (don't forget to chmod +x it) that takes a command

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
34: 
35: 
36: 
#!/usr/bin/env bash


# fck tool path
fckpath=$(readlink -f "$0")
# fck tool dir
dir=$(dirname $fckpath)
script="$dir/fck-cmd/fck-$1.fsx"
shell="$dir/fck-cmd/fck-$1.sh"
cmd="$1"
shift

# packages if needed
if [ ! -d "$dir/fck-cmd/packages" ]
then
pushd "$dir/fck-cmd" > /dev/null
    mono "$dir/fck-cmd/.paket/paket.bootstrapper.exe" --run restore
popd > /dev/null
fi

# script command if it exists
if [ -e $script ]
then
    mono "$dir/fck-cmd/packages/FAKE/tools/FAKE.exe" "$script" -- $@

# shell command if it exists
elif [ -e $shell ]
then
    eval $shell $@

# help
else
pushd "$dir/fck-cmd" > /dev/null
    mono "$dir/fck-cmd/packages/FAKE/tools/FAKE.exe" "$dir/fck-cmd/fck-help.fsx" -- $cmd $@
popd > /dev/null
fi

and the batch version:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30: 
31: 
32: 
33: 
@echo off
set encoding=utf-8

set dir=%~dp0
set cmd=%1
set script="%dir%\fck-cmd\fck-%cmd%.fsx"
set batch="%dir%\fck-cmd\fck-%cmd%.cmd"
shift

set "args="
:parse
if "%~1" neq "" (
  set args=%args% %1
  shift
  goto :parse
)
if defined args set args=%args:~1%

if not exist "%dir%\fck-cmd\packages" (
pushd "%dir%\fck-cmd\\"
"%dir%\fck-cmd\.paket\paket.bootstrapper.exe" --run restore
popd
)

if exist  "%script%" (
"%dir%/fck-cmd/packages/fake/tools/fake.exe" "%script%" -- %args%
) else if exist "%batch%" (
pushd "%dir%\fck-cmd\\"
"%batch%" %cmd% %*
popd
) else (
"%dir%/fck-cmd/packages/fake/tools/fake.exe" "%dir%\fck-cmd\fck-help.fsx" -- %cmd% %*
)

Forget the paket part for now.

The bash take a command argument, and check whether a fck-cmd/fck-$cmd.fsx file exists. If it does, run it ! It also works with shell scripts name fck-$cmd.sh or batch scripts fck-$cmd.cmd to integrate quickly with existing tools.

Fake for faster startups

When F# scripts start to grow big, especially with things like Json or Xml type providers, load time can start to raise above acceptable limits for a cli.

Using Fake to launch scripts takes adventage of it's compilation cache. We get the best of both world:

  • scriptability for quick changes and easy deployment
  • automaticly cached jit compilation for fast startup and execution

We could have written all commands in a single fsx file and pattern maching on the command name, but once we start to have more commands, the script becomes bigger and compilation longer. The problem is also that the pattern matching becomes a friction point in the source control.

FckLib

At some point we have recuring code in the tools. So we can create helper scripts that will be included by command scripts.

For instance parsing the command line is often useful so I created a helper:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
#r "System.Xml.Linq"
#I "../packages/FAKE/tools"
#r "FakeLib.dll"

open Fake
open System

// culture invariant, case insensitive string comparison
let (==) x y = String.Equals(x,y, StringComparison.InvariantCultureIgnoreCase)

open System.Xml.Linq

module CommandLine =
    // get the command line, fck style...
    let getCommandLine() = 
        System.Environment.GetCommandLineArgs() 
        |> Array.toList
        |> List.skipWhile ((<>) "--")
        |> List.tail

    // check whether the command line starts with specified command
    let (|Cmd|_|) str cmdLine =
        match cmdLine with
        | s :: _ when s == str -> Some()
        | _ -> None 

We use the -- to delimit arguments reserved for the script. Since Fake is used to launch scripts, we can also include FakeLib for all the fantastic helpers it contains.

Here is a sample fck-cmd/fck-hello.fsx script that can write hello.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
#load "fcklib/FckLib.fsx"
#r "packages/FAKE/tools/FakeLib.dll"

open FckLib.CommandLine
open Fake

let name =
    match getCommandLine() with
    | name :: _ -> name
    | _ -> "you"

tracefn "Hello %s" name

It uses FakeLib for the tracefn function and FckLib for getCommandLine.

You can call it with (once fck is in your Path environment variable):

1: 
fck hello Santa

Help

A tool without help is just a nightmare, and writing help should be easy.

The last part of fck bash script lanch the fck-help.fsx script:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
#load "fcklib/FckLib.fsx"
open System.IO
open Printf
open FckLib.CommandLine
let root = __SOURCE_DIRECTORY__
let cmd =
    match getCommandLine() with
    | cmd :: _ -> cmd
    | _ -> "help"

let file cmd = sprintf "%s/fck-%s.txt" root cmd
 
let filename = file cmd

if File.Exists filename then
    File.ReadAllText filename
    |> printfn "%s"
else
    printfn "The command %s doesn't exist" cmd 
    printfn ""
    file "help"
    |> File.ReadAllText
    |> printfn "%s" 

This script tries to find a fck-xxx.txt file and display it, or fallbacks to fck-help.txt.

For exemple the help for our fck hello command will be in fck-hello.txt:

1: 
2: 
3: 
4: 
Usage:
fck hello [<name>]

Display a friendly message to <name> or to you if <name> is omitted.

Of course we can the pimp the fck-help.fsx to parse the txt help files and add codes for colors, verbosity etc.

Deployment

Deployment is really easy. We can clone the git repository, and add it to $PATH.

Run the commands, it will automatically restore packages if missing, and lanch the script.

To upgrade to a new version, call fck update, defined in fck-update.sh :

1: 
2: 
3: 
4: 
5: 
6: 
7: 
script=$(readlink -f "$0")
dir=$(dirname $script)

pushd "$dir" > /dev/null
git pull
mono "$dir/.paket/paket.bootstrapper.exe" --run restore
popd > /dev/null

or batch fck-update.cmd:

1: 
2: 
git pull
.paket

Yep, that's that easy

Happy Christmas

Using Santa's elves tools, I hope you won't be stuck at work on xmas eve ! Enjoy !

The full source is on github

val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn
val set : elements:seq<'T> -> Set<'T> (requires comparison)

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.set
val not : value:bool -> bool

Full name: Microsoft.FSharp.Core.Operators.not
namespace Fake
Multiple items
union case ServiceInstallStart.System: ServiceInstallStart

--------------------
namespace System
val x : string
val y : string
Multiple items
type String =
  new : value:char -> string + 7 overloads
  member Chars : int -> char
  member Clone : unit -> obj
  member CompareTo : value:obj -> int + 1 overload
  member Contains : value:string -> bool
  member CopyTo : sourceIndex:int * destination:char[] * destinationIndex:int * count:int -> unit
  member EndsWith : value:string -> bool + 2 overloads
  member Equals : obj:obj -> bool + 2 overloads
  member GetEnumerator : unit -> CharEnumerator
  member GetHashCode : unit -> int
  ...

Full name: System.String

--------------------
String(value: nativeptr<char>) : unit
String(value: nativeptr<sbyte>) : unit
String(value: char []) : unit
String(c: char, count: int) : unit
String(value: nativeptr<char>, startIndex: int, length: int) : unit
String(value: nativeptr<sbyte>, startIndex: int, length: int) : unit
String(value: char [], startIndex: int, length: int) : unit
String(value: nativeptr<sbyte>, startIndex: int, length: int, enc: Text.Encoding) : unit
String.Equals(a: string, b: string) : bool
String.Equals(a: string, b: string, comparisonType: StringComparison) : bool
type StringComparison =
  | CurrentCulture = 0
  | CurrentCultureIgnoreCase = 1
  | InvariantCulture = 2
  | InvariantCultureIgnoreCase = 3
  | Ordinal = 4
  | OrdinalIgnoreCase = 5

Full name: System.StringComparison
field StringComparison.InvariantCultureIgnoreCase = 3
namespace System.Xml
namespace System.Xml.Linq
val getCommandLine : unit -> 'a

Full name: Fck.CommandLine.getCommandLine
type Environment =
  static member CommandLine : string
  static member CurrentDirectory : string with get, set
  static member Exit : exitCode:int -> unit
  static member ExitCode : int with get, set
  static member ExpandEnvironmentVariables : name:string -> string
  static member FailFast : message:string -> unit + 1 overload
  static member GetCommandLineArgs : unit -> string[]
  static member GetEnvironmentVariable : variable:string -> string + 1 overload
  static member GetEnvironmentVariables : unit -> IDictionary + 1 overload
  static member GetFolderPath : folder:SpecialFolder -> string + 1 overload
  ...
  nested type SpecialFolder
  nested type SpecialFolderOption

Full name: System.Environment
Environment.GetCommandLineArgs() : string []
type Array =
  member Clone : unit -> obj
  member CopyTo : array:Array * index:int -> unit + 1 overload
  member GetEnumerator : unit -> IEnumerator
  member GetLength : dimension:int -> int
  member GetLongLength : dimension:int -> int64
  member GetLowerBound : dimension:int -> int
  member GetUpperBound : dimension:int -> int
  member GetValue : [<ParamArray>] indices:int[] -> obj + 7 overloads
  member Initialize : unit -> unit
  member IsFixedSize : bool
  ...

Full name: System.Array
val toList : array:'T [] -> 'T list

Full name: Microsoft.FSharp.Collections.Array.toList
Multiple items
module List

from Microsoft.FSharp.Collections

--------------------
val str : string
val cmdLine : string list
val s : string
union case Option.Some: Value: 'T -> Option<'T>
union case Option.None: Option<'T>
module FckLib
module CommandLine

from FckLib
val name : string

Full name: Fck.name
val getCommandLine : unit -> 'a

Full name: FckLib.CommandLine.getCommandLine


 get the command line, fck style...
val name : string
val tracefn : fmt:Printf.StringFormat<'a,unit> -> 'a

Full name: Fake.TraceHelper.tracefn
namespace System.IO
module Printf

from Microsoft.FSharp.Core
val root : string

Full name: Fck.root
val cmd : string

Full name: Fck.cmd
val cmd : string
val file : cmd:string -> string

Full name: Fck.file
val sprintf : format:StringFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.Printf.sprintf
val filename : string

Full name: Fck.filename
Multiple items
active recognizer File: FileSystemInfo -> Choice<FileInfo,(DirectoryInfo * Collections.Generic.IEnumerable<FileSystemInfo>)>

Full name: Fake.FileHelper.( |File|Directory| )

--------------------
type File =
  static member AppendAllLines : path:string * contents:IEnumerable<string> -> unit + 1 overload
  static member AppendAllText : path:string * contents:string -> unit + 1 overload
  static member AppendText : path:string -> StreamWriter
  static member Copy : sourceFileName:string * destFileName:string -> unit + 1 overload
  static member Create : path:string -> FileStream + 3 overloads
  static member CreateText : path:string -> StreamWriter
  static member Decrypt : path:string -> unit
  static member Delete : path:string -> unit
  static member Encrypt : path:string -> unit
  static member Exists : path:string -> bool
  ...

Full name: System.IO.File
File.Exists(path: string) : bool
File.ReadAllText(path: string) : string
File.ReadAllText(path: string, encoding: Text.Encoding) : string
val printfn : format:TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.Printf.printfn