Jupyter - problems with dotnet-try

Since .NET Try supports F# officially I wanted to switch over my Jupyter notebook instance to use that instead of IFsharp. But I ran into a couple of frustrating issues that I wanted to document in case anyone else hit them and didn't know how to debug the issue.

Issue 1. dotnet-try installs but isn't usable

I followed Scott Hanselman's instructions to install dotnet-try, but when I tried to execute dotnet try jupyter install it seemed as though dotnet-try wasn't installed at all:

$ dotnet tool install dotnet-try --global
You can invoke the tool using the following command: dotnet-try
Tool 'dotnet-try' (version '1.0.19553.4') was successfully installed.

$ dotnet try jupyter install Could not execute because the specified command or file was not found. Possible reasons for this include: * You misspelled a built-in dotnet command. * You intended to execute a .NET Core program, but dotnet-try does not exist. * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.

After a lot of head scratching I dug into the actual docs and learned that both .NET Core 3.0 and .NET Core 2.1 SDKs should be installed, while I only had .NET Core 3.1's SDK So after a quick sudo apt install dotnet-sdk-3.0 dotnet-sdk-2.1 I was successfully able to install the kernel and list it:

$ jupyter kernelspec list
Available kernels:
  .net-csharp    /home/notebook/.local/share/jupyter/kernels/.net-csharp
  .net-fsharp    /home/notebook/.local/share/jupyter/kernels/.net-fsharp
  mit-scheme     /usr/local/share/jupyter/kernels/mit-scheme
  python3        /usr/local/share/jupyter/kernels/python3

Issue 2. Jupyter can't run dotnet-try

However even though it was installed, each time I tried to create a new F# notebook Jupyter would give an error saying that it was unable to connect to the kernel. After taking a quick look at my logs I saw the same error as before!

Feb 09 13:19:11 aviemore jupyter[837]: [I 13:19:11.169 NotebookApp] KernelRestarter: restarting kernel (1/5), new random ports
Feb 09 13:19:11 aviemore jupyter[837]: Could not execute because the specified command or file was not found.
Feb 09 13:19:11 aviemore jupyter[837]: Possible reasons for this include:
Feb 09 13:19:11 aviemore jupyter[837]: * You misspelled a built-in dotnet command.
Feb 09 13:19:11 aviemore jupyter[837]: * You intended to execute a .NET Core program, but dotnet-try does not exist.
Feb 09 13:19:11 aviemore jupyter[837]: * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.
Feb 09 13:19:14 aviemore jupyter[837]: [I 13:19:14.180 NotebookApp] KernelRestarter: restarting kernel (2/5), new random ports
Feb 09 13:19:14 aviemore jupyter[837]: Could not execute because the specified command or file was not found.
Feb 09 13:19:14 aviemore jupyter[837]: Possible reasons for this include:
Feb 09 13:19:14 aviemore jupyter[837]: * You misspelled a built-in dotnet command.
Feb 09 13:19:14 aviemore jupyter[837]: * You intended to execute a .NET Core program, but dotnet-try does not exist.
Feb 09 13:19:14 aviemore jupyter[837]: * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH. 
I restarted the service, the server, checked and it took a while to realise the root cause. What happened was, even though dotnet-try was in PATH when I switched over to my jupyter use, it wasn't the case when I ran it via systemd. It seems that /etc/profile - which runs all the scripts in /etc/profile.d, one of which adds the ~/.dotnet/tools dir to PATH - is not used when systemd starts the service. I don't know the correct way to address this, but I wrote a quick script to setup PATH correctly and added an EnvironmentFile to the [Service] section my notebook.service file to use it:
After I set this up and restarted notebook.service it was able to access dotnet-try and spin up the F# kernel correctly.

F# - Web applications with Angular and Giraffe

tl;dr - If you want a barebones .NET web app written in F# with Angular and Giraffe all hooked up already then clone my giraffe-ng repo on GitHub as a starting point. If you want to know the steps involved in setting this up, then read on.

The Angular CLI is a powerful and easy way to build rich web applications. However in some cases you might want to use something other than the NodeJS backend that it provides by default. This is a little guide to show how an Angular application can be created with a backend powered by F# using the Giraffe framework

To begin with we need to ensure that Angular CLI and .NET Core SDK 2.2 are installed and in our PATH:

$ dotnet --version
$ ng --version
     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|

Angular CLI: 7.0.4
Node: 9.11.2
OS: linux x64

Package                      Version
@angular-devkit/architect    0.10.4
@angular-devkit/core         7.0.4
@angular-devkit/schematics   7.0.4
@schematics/angular          7.0.4
@schematics/update           0.10.4
rxjs                         6.3.3
typescript                   3.1.3

First create the folder, the  .NET project and install the Giraffe and Microsoft.AspNetCore.App packages:

  $ mkdir giraffe-ng
  $ cd giraffe-ng 
  $ dotnet new console -lang F#
  $ dotnet add package Giraffe --version 3.4.0
  $ dotnet add package Microsoft.AspNetCore.App --version 2.2.0 

We'll now create a simple landing page using Giraffe's own view engine just to test everything is working.

  let index = 
      html [] [
          head [] [
              title [] [ str "Giraffe!" ]
          body [] [
              h1 [] [ str "Hello!" ]
              p [] [ str "A test of Giraffe and .NET Core"]
  let webApp =
      choose [ route "/" >=> (index |> renderHtmlDocument |> htmlString) ]
  let configureApp (app : IApplicationBuilder) =
  let configureServices (services : IServiceCollection) =
      services.AddGiraffe() |> ignore
  let main _ =
          .Configure(Action configureApp)

We can test that this works nicely by going to http://localhost:5000 - the reason the port is important will be apparent later:

Looks good, so now we can setup our frontend, so we'll use the Angular CLI:

$ ng new frontend

And we'll check out http://localhost:4200 to make sure it's working:

What we want to do is take the Angular application and serve it with out F#\Giraffe app via the route "/app" - so that viewing http://localhost:5000 will serve the Angular app that was previously served by NodeJS on http://localhost:4200. We'll start off by making sure our application is setup to find all its resources under the /app route.

$ ng build --base-href /app/
We'll next change our app so that it'll serve this route:
let configureApp (app : IApplicationBuilder) =
              FileProvider = new PhysicalFileProvider(
                      Path.Combine(Directory.GetCurrentDirectory(), "frontend", "dist")),
                      RequestPath = PathString("/app")))

We now need to make sure the files in the frontend/dist directory is copied to the .NET application's bin directory when it's built by adding the following to our .fsproj file:
    <Content Include="frontend/dist/*.*">

Next we can change the definition of our index view so that our Angular scripts and application root are loaded:

  let ngApp = tag "app-root"  [] []
  let ngScripts = 
      [ "runtime.js"; "polyfills.js"; "styles.js"; "vendor.js"; "main.js" ]
      |> List.map (function js -> script [ _type "text/javascript"; _src js ] [] )
  let index = 
      html [ _lang "en" ] [
          head [] [
              meta [ _charset "utf-8"; _name "viewport"; _content "width=device-width, initial-scale=1" ] 
              title [] [ str "Angular + Giraffe" ]
              ``base`` [ _href "/app/" ]
              link [ _rel "icon"; _type "icon"; _href "favicon.ico" ]
          body [] 
              (ngApp :: ngScripts)

Now if we run the Giraffe app and navigate to http://localhost:5000 we should see our Angular application:

And there we have it - we set up an application with an F# and Giraffe powered backend and an Angular 7 frontend. Extending this to have the Angular app fetch data from a Giraffe service is pretty straight-forward. As an example if we want our page to display a list of users it has requested from the backend, we can add the following to our Program.fs:

type User = { login:string; email:string }

let userList:(User list) = [ 
  { login = "sean"; email = "sean@example.com" }
  { login = "lucka"; email = "lucka@example.com" }
  { login = "alfie"; email = "alfie@example.com" }
  { login = "ivy"; email = "ivy@example.com" }

Then we can add a new “users” route to our webapp to serve this data:

let webApp =
  choose [
      route "/"       >=> (index |> renderHtmlDocument |> htmlString) 
      route "/users" >=> (json userList)

Then in our main app.component.ts we'll add a type to represent the User

interface User {
    login: string;
    email: string;

And we can now modify our AppComponent so that it hits this service

  export class AppComponent implements OnInit {
    title = 'frontend';
    public users: User[] = [];
    constructor(private http: HttpClient) {
    ngOnInit(): void {
      this.http.get('/users').subscribe(users => {
        this.users = users;

And we can modify the component's template to render this

    <li *ngFor="let user of users">
      <a href="mailto:{{user.email}}">{{user.login}}</a>
So there we are, an Angular frontend served by Giraffe and F#! Just for reference the completed app is available on GitHub under my "giraffe-ng" repo.

F# - Polymorphic parameter overflow

I was mucking around in F# over lunch and started thinking about how it assigns names to type variables (see "Automatic Generalization" of the Type Inference page of the F# docs for more info). By way of introduction, let's create a function foo that takes a single parameter without an explicitly defined type and see what its signature looks like:
> let foo x = x;;
val foo : x:'a -> 'a
So the parameter x is assigned the type variable 'a - which makes sense, the first unknown type gets named after the first letter of the alphabet. And of course it follows that if we have a function with two parameters the second type is called ...
> let foo x y = (x,y);;
val foo : x:'a -> y:'b -> 'a * 'b
... 'b! Ok now what happens when we've got a function with a ridiculous amount of parameters? We'll run out of lower-case letters eventually. When that happens do start using upper-case letters? Non-ASCII? Something else? I quickly hacked together a python script to generate functions with arbitrary parameters, created one with 40 and pasted it into the F# REPL and ...
> let foo x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x20 x21 x22 x23 x24 x25 x26 x27 x28 x29 x30 x31 x32 x33 x34 x35 x36 x37 x38 x39 = (x0,x1,x2,x- 3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15,x16,x17,x18,x19,x20,x21,x22,x23,x24,x25,x26,x27,x28,x29,x30,x31,x32,x33,x34,x35,x36,x37,x38,x39);;
val foo :
  x0:'a ->
    x1:'b ->
      x2:'c ->
        x3:'d ->
          x4:'e ->
            x5:'f ->
              x6:'g ->
                x7:'h ->
                  x8:'i ->
                    x9:'j ->
                      x10:'k ->
                        x11:'l ->
                          x12:'m ->
                            x13:'n ->
                              x14:'o ->
                                x15:'p ->
                                  x16:'q ->
                                    x17:'r ->
                                      x18:'s ->
                                        x19:'t ->
                                          x20:'a1 ->
                                            x21:'a2 ->
                                              x22:'a3 ->
                                                x23:'a4 ->
                                                  x24:'a5 ->
                                                    x25:'a6 ->
                                                      x26:'a7 ->
                                                        x27:'a8 ->
                                                          x28:'a9 ->
                                                            x29:'a10 ->
                                                              x30:'a11 ->
                                                                x31:'a12 ->
                                                                  x32:'a13 ->
                                                                    x33:'a14 ->
                                                                      x34:'a15 ->
                                                                        x35:'a16 ->
                                                                          x36:'a17 ->
                                                                            x37:'a18 ->
                                                                              x38:'a19 ->
                                                                                x39:'a20 ->
                                                                                  'a *
                                                                                  'b *
                                                                                  'c *
                                                                                  'd *
                                                                                  'e *
                                                                                  'f *
                                                                                  'g *
                                                                                  'h *
                                                                                  'i *
                                                                                  'j *
                                                                                  'k *
                                                                                  'l *
                                                                                  'm *
                                                                                  'n *
                                                                                  'o *
                                                                                  'p *
                                                                                  'q *
                                                                                  'r *
                                                                                  's *
                                                                                  't *
                                                                                  'a1 *
                                                                                  'a2 *
                                                                                  'a3 *
                                                                                  'a4 *
                                                                                  'a5 *
                                                                                  'a6 *
                                                                                  'a7 *
                                                                                  'a8 *
                                                                                  'a9 *
                                                                                  'a10 *
                                                                                  'a11 *
                                                                                  'a12 *
                                                                                  'a13 *
                                                                                  'a14 *
                                                                                  'a15 *
                                                                                  'a16 *
                                                                                  'a17 *
                                                                                  'a18 *
                                                                                  'a19 *
So 'a up to 't ... but then numbered 'as after that. I've no idea why it stops at 't, then just counts up from 'a. Interestingly this is the case with F# under Mono on Linux and under the official MS runtime on VS2017 on Windows. OCaml, which F# is heavily influenced by, does not exhibit this behaviour - it refers to the unknown types as a..z, a1..z1, a2...z2, etc.

That's reasonbaly interesting and all, but since I had already written a script to generate these F# files I figured I'd keep trying with more and more parameters to see what happens. After adding a hundred parameters at a time I soon hit an interesting warning between 400 and 500 params:

Warning: line too long, ignoring some characters
It turned out we hit some line-length restriction in either the parser or the readline equivalent that fsi.exe uses. A bit of experimentation later I found that when the function had 429 parameters it parsed just fine ... but 430 parameters caused the warning.

429 parameters: 4093 characters
430 parameters: 4104 characters

Minor sidebar - I was going to just say we hit a restriction where 4096 bytes is the max line length, but obviously 1 character isn't necessarily 1 byte if we're using some Unicode representation, and a nice way to check that we're using Unicode is to punch in identifiers that would definitely be in different encodings in an ASCII representation - I used Georgian and Czech characters:
> let სამი = 3;;
val სამი : int = 3

> let čtyři = 4;;
val čtyři : int = 4

> სამი + čtyři;;
val it : int = 7
When I changed my program to start each parameter with the Georgian character "ა" (i.e. let bar ა0 ა1... etc) which cannot be represented in a single byte in UTF-8 we hit the error after only 162 parameters. So it's not a character limit, but a buffer of bytes that we filled - and the size of the buffer is ... uh 4096 bytes.

Since I wanted to pull this thread until I hit some logical conclusion I tweaked my program to split the insanely long lines so they wouldn't hit this error and continued to add more parameters. The next issue I encountered was after trying a function with a couple thousand parameters, where I encountered the following error:

/home/sean/stdin(6636,1): error FS0039: The value or constructor 'x1328' is not defined. Maybe you want one of the following:
So it seems we've bumped up into another limit - somehow 1327 parameters is ok but F# loses track of the 1328th one, thinking it doesn't exist. Is this a bug? Probably, there should maybe be a more helpful error message. Is it important? Probably not, if your code contains a function with upwards of a thousand parameters then this is the least of your problems.

IIS - Annoying error with .NET Core

I deployed a .NET Core application recently into an environment which previously had only run .NET Framework apps, but got a tricky HTTP Error 500.19 - Internal Server Error suggesting there was a problem with my Web.config:

What was also peculiar was that when I tried to enable stdout logging using the IIS UI I got an error "There was an error while performing this operation" with blank "Details" and "Error" but with the "Filename" set to the path to my Web.config. This was true for any of the configuration sections:

However after a lot of head scratching and googling turned up nothing my colleague realised that this environment was missing the Windows Server Hosting bundle, available from the .NET Core downloads section under the "Other Windows Downloads" section. Once this was installed everything worked as expected.