3DICC / SqueakNim

  # SqueakNim - Squeak and Nim interop via FFI

  Nim can produce dynamically loaded libraries that follow the C conventions. 
  Squeak/Pharo has over the years developed several FFI mechanisms, and we can 
  also make VM plugins, but for now this code uses the old "true and tested" 

  Squeak FFI.

  ## How to use it

  You write a Nim module and use some macros to generate a .st file on the side
  containing the Smalltalk glue code. When you compile the Nim code the .st file
  is automatically generated. You then file that into the image.

  See full example in tests directory. Here is how to try it:
  
  - Install Nim, latest. See nim-lang.org.
  - Clone squeaknim, its just a single source file really.
  - Install FFI into your Squeak/Nim image. Old one.
  - Compile the Test1.nim file using "nim c Test1.nim". Should give you Test1.st and libTest1.so/dll.
  - Make sure the compiled libTest1.so/dll is loadable by the VM (copy to working dir for example)
  - File in Test1.st that was generated.
  - File in Test1-Test.st that has a SUnit test for the above glue code.
  - Run in SUnit. Bask in green glory, or cry.
  
  The FFI should be the "good old" one - via Configurations in Pharo seems to work fine. Via SqueakMap worked in Squeak 5, I think I had to use "head".

  ## Squeak FFI
  

  What follows is a text written by Sean DeNigris describing the FFI. You can
  also see the code in the client image of course, with tests and class comments

  etc.

  ### How does FFI work?

  
  Technically what happens is:
  
  * you define what the interface is - the parameters, types etc.
  * when you make the call, the FFI logic assembles the data from the Squeak
  Objects into the proper structures according to the routine calling
  conventions for your architecture, and of course manages the return values.
  So no magic but perhaps just a little assembler in the plugin to properly
  deal with all the registers and condition flags.
  
  How do I use it?
  
  1. make a method (whose structure is similar to a named primitive method)
  
  Example: 
  
  ```

  system: aString

    "Some kind of comment"
  
  	<apicall: long 'system' (char*) module: 'libSystem.dylib'>
  	^self externalCallFailed.
  ```
  

  `system: aString` above is the Smalltalk signature. Each named parameter is
  separated with a keyword. Smalltalk messages are normally camelCase, so if
  a nim proc was called `drink(bottles: int, brand: string)` then we could
  typically turn that into `drinkBottles: bottles brand: brand`. The unique
  (per class) so called `selector` (=name) of such a method in Smalltalk would
  be `drinkBottles:brand:`. Thus Nim overloading would not work (but we don't
  need that here).

  
  Then comes the so called pragma that describes the C function to call:
  
  ```
  	<apicall: long 'system' (char*) module: 'libSystem.dylib'>
  ```

  Function specification should be the first line in the method and enclosed in 
  angle brackets: < > containing:

  1. Calling Convention, either apicall: (Pascal convention) or cdecl: (C convention)
  	- Mac - use either one
  	- Unix - use cdecl
  	- Windows - use apical
  2. Return Type (see types)
  3. External Function Name (literal string)
  4. Argument Types (a literal array)
  5. Module - "module: " + [filename of the external library (literal string)] (see below).

  
  ```
  	^self externalCallFailed.
  ```
  

  Failure handler:

  * normal smalltalk code
  * executed if the linking to or calling the external function fails
  * API calls don't know how to communicate failure like Squeak primitives do, so:
    * it does not tell you whether the external function succeeded
    * the most common code is simply '^self externalCallFailed.'
  
  Argument Types must be names of ExternalTypes, either:

  		- atomic types (see ExternalType class>>initializeFFIConstants and ExternalType class>>initializeAtomicTypes):
  			void
  			bool
  			byte (unsigned)
  			sbyte (signed)
  			ushort (16-bit unsigned)
  			short (16-bit signed)
  			ulong (32-bit unsigned)
  			long (32-bit signed)
  			ulonglong (64-bit unsigned)
  			longlong (64-bit signed)
  			char (unsigned)
  			schar (signed)
  			float (single-precision float)
  			double (double-precision float)
  
  Structure Types [4]

  * subclass of ExternalStructure
  * class>>fields that returns an array of field descriptions (see below), example:
  			fields
  				^#((red   'byte')(green 'byte')(blue  'byte'))           
  * class>>initialize which includes "self defineFields" (which must be called before using the class)
  * refer to as MyExternalStructure* (to indicate that the argument or return is a pointer to that structure)

  
  Field description [4]

  * 2-element array (or three but that does something else, I'm not sure what):
  	* first element is the field name
  	* second is the type

  
  Module Name 
  - depends on the platform
  	- Mac
  		- pre Snow Leopard: flexible, can eliminate leading lib or extension e.g. 'libc.dylib' becomes 'libc', 'c.dylib', or 'c'
  		- Snow Leopard
  			- file name must be exact including extension (unless Info.plist is altered as in 'Library Location' below)
  		- With pre-mach-o VMs
  			- For Classic applications, use 'InterfaceLib'
  			- For Carbon libs, use 'CarbonLib'
  

  Module Location, where the external library file lives
  - depends on the platform
  	- Mac
  		- pre Snow Leopard
  			- checks VM path and common library paths
  		- Snow Leopard
  			- only looks in VM bundle's Resources file, you must either [5]:
  				- store all external libraries there
  				- ln -s path/to/library path/to/VM/Resources/library_name
  				- Change the VM's Info.plist "SqueakPluginsBuiltInOrLocalOnly" key from "true" to "false."

  Caveats

  - security

  	- malicious users could call arbitrary functions in the OS e.g. 
  	  "format c:" from "system.dll" [7]

  	- VMs do not protect against buffer overflow from bad parameters [8]:
  		"this would require an attacker to execute arbitrary Smalltalk 
  		code on your server. Of course if they can do that they own you 

  		anyway, especially if you allow FFi or use the OSProcess plugin"
  		 - John McIntosh

  
  * difficulty

  	- if you make a mistake you'll not drop into the debugger but Squeak
  	  will just crash [2]
  	- If you crash Squeak when it is running the garbage collector, then
            you know your FFI code is leaking bits into object memory [2]

  ### What do I need to use FFI with Squeak?

  
  You need the FFI plugin, which is included with most VM's as of Squeak 3.6
  or so.
  
  You can also build the plugin yourself. See VMMaker.
  
  References:

  1. http://wiki.squeak.org/squeak/1414
  2. http://wiki.squeak.org/squeak/2424
  3. http://wiki.squeak.org/squeak/5716
  4. http://wiki.squeak.org/squeak/2426
  5. http://forum.world.st/squeak-dev-Alien-Squeak-FFI-issues-on-Snow-Leopard-td85608.html
  6. http://wiki.squeak.org/squeak/5846
  7. http://forum.world.st/FFI-Callbacks-td54056.html#a54073
  8. http://forum.world.st/Security-td99624.html#a99635: