first somewhat working version of SqueakNim

Andreas Rumpf
1 parent 24d7cba1
Showing 4 changed files with 212 additions and 16 deletions
README.md
src/squeaknim.nim
testlib/README.md
tests/test1.nim
 # NimSqueak - Squeak and Nim interop via FFI
  
-Nim can produce dynamically loaded libraries that follow the C conventions. Squeak/Pharo has over the years developed several FFI mechanisms, and we can also make VM plugins, but for now this code uses the old "true and tested" Squeak FFI that Terf uses today for all its OpenGL calls.
+Nim can produce dynamically loaded libraries that follow the C conventions. 
+Squeak/Pharo has over the years developed several FFI mechanisms, and we can 
+also make VM plugins, but for now this code uses the old "true and tested" 
+Squeak FFI that Terf uses today for all its OpenGL calls.
  
-I also wrote an article when I experimented: http://goran.krampe.se/2014/11/03/squeak-to-nim/
+I also wrote an article when I experimented: 
+http://goran.krampe.se/2014/11/03/squeak-to-nim/
  
-In the testlib directory we have the files from that article. See README.md there for a concrete rundown on how to generate the .st file.
+In the testlib directory we have the files from that article. See README.md 
+there for a concrete rundown on how to generate the .st file.
  
 ## Squeak FFI
  
-What follows is a text written by Sean DeNigris describing the FFI. You can also see the code in the client image of course, with tests and class comments etc. And then we have the actual SqueakFFIPrims plugin (VM level code) that you can find in the terf-client-vm repo.
+What follows is a text written by Sean DeNigris describing the FFI. You can
+also see the code in the client image of course, with tests and class comments
+etc. And then we have the actual SqueakFFIPrims plugin (VM level code) that
+you can find in the terf-client-vm repo.
  
 ## How does FFI work?
  
@@ -35,14 +43,21 @@ system: aString
 	^self externalCallFailed.
 ```
  
-`system: aString` above is the Smalltalk signature. Each named parameter is separated with a keyword. Smalltalk messages are normally camelCase, so if a nim proc was called `drink(bottles: int, brand: string)` then we could typically turn that into `drinkBottles: bottles brand: brand`. The unique (per class) so called `selector` (=name) of such a method in Smalltalk would be `drinkBottles:brand:`. Thus Nim overloading would not work (but we don't need that here).
+`system: aString` above is the Smalltalk signature. Each named parameter is
+separated with a keyword. Smalltalk messages are normally camelCase, so if
+a nim proc was called `drink(bottles: int, brand: string)` then we could
+typically turn that into `drinkBottles: bottles brand: brand`. The unique
+(per class) so called `selector` (=name) of such a method in Smalltalk would
+be `drinkBottles:brand:`. Thus Nim overloading would not work (but we don't
+need that here).
  
 Then comes the so called pragma that describes the C function to call:
  
 ```
 	<apicall: long 'system' (char*) module: 'libSystem.dylib'>
 ```
-Function specification should be the first line in the method and enclosed in angle brackets: < > containing:
+Function specification should be the first line in the method and enclosed in 
+angle brackets: < > containing:
 1. Calling Convention, either apicall: (Pascal convention) or cdecl: (C convention)
 	- Mac - use either one
 	- Unix - use cdecl
@@ -116,15 +131,19 @@ Module Location, where the external library file lives
 				- Change the VM's Info.plist "SqueakPluginsBuiltInOrLocalOnly" key from "true" to "false."
 Caveats
 - security
-	- malicious users could call arbitrary functions in the OS e.g.  "format c:" from "system.dll" [7]
+	- malicious users could call arbitrary functions in the OS e.g. 
+	  "format c:" from "system.dll" [7]
 	- VMs do not protect against buffer overflow from bad parameters [8]:
 		"this would require an attacker to execute arbitrary Smalltalk 
 		code on your server. Of course if they can do that they own you 
-		anyway, especially if you allow FFi or use the OSProcess plugin" - John McIntosh
+		anyway, especially if you allow FFi or use the OSProcess plugin"
+		 - John McIntosh
  
 * difficulty
-	- if you make a mistake you'll not drop into the debugger but Squeak will just crash [2]
-	- If you crash Squeak when it is running the garbage collector, then you know your FFI code is leaking bits into object memory [2]
+	- if you make a mistake you'll not drop into the debugger but Squeak
+	  will just crash [2]
+	- If you crash Squeak when it is running the garbage collector, then
+          you know your FFI code is leaking bits into object memory [2]
  
 What do I need to use FFI with Squeak?
  
@@ -138,8 +157,7 @@ References:
 [2] http://wiki.squeak.org/squeak/2424
 [3] http://wiki.squeak.org/squeak/5716
 [4] http://wiki.squeak.org/squeak/2426
-[5]
-http://forum.world.st/squeak-dev-Alien-Squeak-FFI-issues-on-Snow-Leopard-td85608.html
+[5] http://forum.world.st/squeak-dev-Alien-Squeak-FFI-issues-on-Snow-Leopard-td85608.html
 [6] http://wiki.squeak.org/squeak/5846
 [7] http://forum.world.st/FFI-Callbacks-td54056.html#a54073
 [8] http://forum.world.st/Security-td99624.html#a99635:
+
+import macros, strutils
+
+const
+  pragmaPos = 4
+  paramPos = 3
+  intType = when sizeof(int) == 8: "longlong" else: "long"
+  uintType = when sizeof(int) == 8: "ulonglong" else: "ulong"
+
+var
+  dllName {.compileTime.}: string = "SqueakNimTest"
+  stCode {.compileTime.}: string = ""
+
+template setModulename*(s: string) =
+  ## Sets the DLL name. This is also used to set the 'category' in the generated
+  ## classes.
+  static:
+    dllName = s
+
+template writeExternalLibrary*() =
+  static:
+    addf(stCode, """ExternalLibrary subclass: #$1
+    instanceVariableNames: ''
+    classVariableNames: ''
+    poolDictionaries: ''
+    category: '$1'!
+
+!$1 class methodsFor: 'primitives' stamp: 'SqueakNim'!
+""", capitalize(dllName))
+
+template writeSmallTalkCode*(filename: string) =
+  ## You need to invoke this template to write the produced SmallTalk code to
+  ## a file.
+  static:
+    writeFile(filename, stCode)
+
+proc mapTypeToC(symbolicType: NimNode): string {.compileTime.} =
+  let t = symbolicType.getType
+  if symbolicType.kind == nnkSym and t.typeKind == ntyObject:
+    return $symbolicType
+  case t.typeKind
+  of ntyTuple:
+    result = $t
+    when false:
+      let tt = if t.kind == nnkBracketExpr: t else: t.getType
+      expectKind t, nnkBracketExpr
+      result = "struct {"
+      for i in 0 .. < tt.len:
+        result.addf "$# Field$#;\n", mapTypeToC(tt[i]), $i
+      result.add "}"
+  of ntyArray, ntyArrayConstr:
+    # implement this!
+    result = "XXX"
+  of ntyOpenArray:
+    result = mapTypeToC(t[1]) & "* " & intType
+  of ntyPtr, ntyVar:
+    expectKind t, nnkBracketExpr
+    result = mapTypeToC(t[1]) & "*"
+  of ntyCString: result = "char*"
+  of ntyInt: result = intType
+  of ntyInt8: result = "sbyte"
+  of ntyInt16: result = "short"
+  of ntyInt32: result = "long"
+  of ntyInt64: result = "longlong"
+  of ntyUInt: result = uintType
+  of ntyUInt8: result = "ubyte"
+  of ntyUInt16: result = "ushort"
+  of ntyUInt32: result = "ulong"
+  of ntyUInt64: result = "ulonglong"
+  of ntyFloat, ntyFloat64: result = "double"
+  of ntyFloat32: result = "float"
+  of ntyBool, ntyChar, ntyEnum: result = "char"
+  else: quit "Error: cannot wrap to Squeak " & treeRepr(t)
+
+macro exportSt*(className: string; body: stmt): stmt =
+  # generates something like:
+
+  # system: aString
+  #"Some kind of comment"
+  #
+  #   <apicall: long 'system' (char*) module: 'libSystem.dylib'>
+  #   ^self externalCallFailed.
+  result = body
+  result[pragmaPos].add(ident"exportc", ident"dynlib", ident"cdecl")
+  let params = result[paramPos]
+  let procName = $result[0]
+  var st = procName
+  #echo treeRepr params
+  if params.len > 1:
+    expectKind params[1], nnkIdentDefs
+    let ident = $params[1][0]
+    if ident.len > 1:
+      st.add(ident.capitalize & ": " & ident)
+    else:
+      st.add(": " & ident)
+  # return type:
+  var apicall = "<cdecl: " & mapTypeToC(params[0]) & " '" &
+               procName & "' ("
+  var counter = 0
+  # parameter types:
+  for i in 1.. <params.len:
+    let param = params[i]
+    let L = param.len
+    for j in 0 .. param.len-3:
+      let name = param[j]
+      let typ = param[L-2]
+      if counter > 0:
+        apicall.add(" ")
+        st.addf(" $1: $1", name)
+      apicall.add(mapTypeToC(typ))
+      inc counter
+  apicall.add(") module: '" & dllName & "'>\n" &
+              " ^self externalCallFailed.\n")
+  stCode.add(st & "\n\"Generated by NimSqueak\"\n" & apicall)
+
+macro wrapObject*(name: string; typ: stmt): stmt =
+  ## Declares a SmallTalk wrapper class.
+  let name = name.strVal.capitalize
+  var t = typ.getType()
+  if t.typeKind == ntyTypeDesc:
+    expectKind t, nnkBracketExpr
+    t = t[1]
+
+  if t.kind != nnkObjectTy: t = t.getType
+  expectKind t, nnkObjectTy
+  t = t[1]
+  expectKind t, nnkRecList
+  var fields = ""
+  for i in 0.. < t.len:
+    expectKind t[i], nnkSym
+    fields.addf "($# '$#')\n", $t[i], mapTypeToC(t[i])
+
+  let st = """ExternalStructure subclass: #$1
+    instanceVariableNames: ''
+    classVariableNames: ''
+    poolDictionaries: 'FFIConstants'
+    category: '$2'!
+
+$1 class
+   instanceVariableNames: ''!
+
+  !$1 class methodsFor: 'field definition' stamp: 'SqueakNim'!
+  fields
+   ^#(
+   $3
+   )! !
+
+  $1 compileFields!
+  $1 defineFields.
+
+""" % [name, dllName, fields]
+  stCode.add(st)
+  result = newStmtList()
+
-The file to generate given testlib.nim would be `Testlib.st`. This is the class old "chunk format" in which the content is separated by `!` marks, in a rather oddish way, but anyway.
+The file to generate given testlib.nim would be `Testlib.st`. This is the class 
+old "chunk format" in which the content is separated by `!` marks, in a rather 
+oddish way, but anyway.
  
 Let's go through it:
  
 ```
 'From Nim on 18 February 2015 at 11:15:50 pm'!
 ```
-Above first chunk is just a String literal with a timestamp, that chunk can be omitted.
+Above first chunk is just a String literal with a timestamp, that chunk can
+be omitted.
  
 ```
 ExternalLibrary subclass: #Testlib
@@ -14,9 +17,12 @@ ExternalLibrary subclass: #Testlib
 	poolDictionaries: ''
 	category: 'Nim'!
 ```
-Then follows the class declaration. Obviously we need to make sure it says `Testlib` and the category will equal the package in Squeak that this class will end up in. For now, let's hard code it to `Nim`.
+Then follows the class declaration. Obviously we need to make sure it says
+`Testlib` and the category will equal the package in Squeak that this class
+will end up in. For now, let's hard code it to `Nim`.
  
-Then we can see that there is a "whitespace only" chunk, since the next part starts with `!`.
+Then we can see that there is a "whitespace only" chunk, since the next part
+starts with `!`.
  
 ```
 !Testlib class methodsFor: 'primitives' stamp: 'gk 11/2/2014 13:05'!
+
+import squeaknim
+
+type
+  MyFloat = float32
+  Vector3 = object
+    x, y, z: MyFloat
+
+setModulename "urhonimo"
+
+wrapObject("Vector3", Vector3)
+
+writeExternalLibrary()
+
+proc foo(a, b: Vector3; c: openArray[int]): cstring {.exportSt: "bar".} =
+  result = "some string here"
+
+writeSmallTalkCode("test1.st")