libpython-clj2.python
Python bindings for Clojure. This library dynamically finds the installed python, loads the shared library and allows Clojure users to use Python modules as if they were Clojure namespaces.
Example:
user> (require '[libpython-clj2.python :as py])
nil
user> (py/initialize!)
;; ... (logging)
:ok
user> (def np (py/import-module "numpy"))
#'user/np
user> (py/py. np linspace 2 3 :num 10)
[2. 2.11111111 2.22222222 2.33333333 2.44444444 2.55555556
2.66666667 2.77777778 2.88888889 3. ]
$a
macro
($a item attr & args)
Call an attribute of an object using automatic detection of the python kwargs. Keywords must be compile time constants. So this won't work with 'apply'. On the other hand, building the positional and kw argmaps happens at compile time as opposed to at runtime. The attr name can be a symbol.
$c
macro
($c item & args)
Call an object using automatic detection of the python kwargs. Keywords must be compile time constants. So this won't work with 'apply'. On the other hand, building the positional and kw argmaps happens at compile time as opposed to at runtime.
add-module
(add-module modname)
Add a python module. This can create a module if it doesn't exist.
afn
(afn item attr & args)
Call an attribute of an object. Arguments are passed in positionally. Any keyword arguments are paired with the next arg, gathered, and passed into the system as *kwargs.
Not having an argument after a keyword is an error.
attr-type-map
(attr-type-map pyobj)
Return a map of attr name to python-type of the attribute
call-attr
(call-attr pyobj attname & args)
Call an attribute on a python object using only positional arguments
call-attr-kw
(call-attr-kw pyobj attname args kw-list)
Call an attribute passing in both positional and keyword arguments.
cfn
(cfn item & args)
Call an object. Arguments are passed in positionally. Any keyword arguments are paired with the next arg, gathered, and passed into the system as *kwargs.
Not having an argument after a keyword argument is an error.
create-class
(create-class name bases cls-hashmap)
Create a new class object. Any callable values in the cls-hashmap
will be presented as instance methods. If you want access to the
'this' object then you must use make-instance-fn
.
Example:
user> (require '[libpython-clj2.python :as py])
nil
user> (def cls-obj (py/create-class
"myfancyclass"
nil
{"__init__" (py/make-instance-fn
(fn [this arg]
(py/set-attr! this "arg" arg)
;;If you don't return nil from __init__ that is an
;;error.
nil))
"addarg" (py/make-instance-fn
(fn [this otherarg]
(+ (py/get-attr this "arg")
otherarg)))}))
#'user/cls-obj
user> cls-obj
__no_module__.myfancyclass
user> (def inst (cls-obj 10))
#'user/inst
user> (py/call-attr inst "addarg" 10)
20
def-unpack
macro
(def-unpack symbols input)
Unpack a set of symbols into a set of defs. Useful when trying to match Python idioms - this is definitely not idiomatic Clojure.
Example:
user> (py/def-unpack [a b c] (py/->py-tuple [1 2 3]))
#'user/c
user> a
1
user> b
2
user> c
3
from-import
macro
(from-import module-path item & args)
Support for the from a import b,c style of importing modules and symbols in python. Documentation is included.
has-attr?
(has-attr? pyobj att-name)
Return true if this python object has this attribute.
has-item?
(has-item? pyobj item-name)
Return true if the python object has an item. Calls hasitem.
import-as
macro
(import-as module-path varname)
Import a module and assign it to a var. Documentation is included.
import-module
(import-module modname)
Import a python module. Module entries can be accessed via get-attr.
initialize!
(initialize! & {:keys [windows-anaconda-activate-bat library-path no-io-redirect?], :as options})
Initialize the python library. If library path is not provided, then the system attempts to execute a simple python program and have python return system info.
Note: all of the options passed to initialize!
may now be provided in
a root-level python.edn
file. Example:
;; python.edn
{:python-executable "/usr/bin/python3.7"
:python-library-path "/usr/lib/libpython3.7m.so"
:python-home "/usr/lib/python3.7"
:python-verbose true}
or, using a local virtual environment:
;; python.edn
{:python-executable "env/bin/python"}
Additionaly the file can contain two keys which can can refer to custom hooks to run code just before and just after python is initialised. Typical use case for this is to setup / verify the python virtual enviornment to be used.
:pre-initialize-fn my-ns/my-venv-setup-fn!
:post-initialize-fn my-ns/my-venv-validate-fn!
A :pre-initialize-fn could for example shell out and setup a python virtual enviornment.
The :post-initialize-fn can use all functions from ns libpython-clj2.python
as libpython-clj is initialised alreday and could for example be used to validate
that later needed libraries can be loaded via calling import-module
.
The file MUST be named python.edn
and be in the root of the classpath.
With a python.edn
file in place, the initialize!
function may be called
with no arguments and the options will be read from the file. If arguments are
passed to initialize!
then they will override the values in the file.
Returns either :ok
in which case the initialization completed successfully or
:already-initialized
in which case we detected that python has already been
initialized via Py_IsInitialized
and we do nothing more.
Options:
:library-path
- Library path of the python library to use.:program-name
- Optional -- will show up in error messages from python.:no-io-redirect?
- True if you don't want python stdout and stderr redirection to out and err.:python-executable
- The python executable to use to find system information.:python-home
- Python home directory. The system first uses this variable, then the environment variable PYTHON_HOME, and finally information returned from python system info.:signals?
- defaults to false - true if you want python to initialized signals. Be aware that the JVM itself uses quite a few signals - SIGSEGV, for instance - during it's normal course of operation. For more information see:
is-instance?
(is-instance? py-inst py-cls)
Return true if inst is an instance of cls. Note that arguments
are reversed as compared to instance?
make-callable
(make-callable ifn options)
(make-callable ifn)
Make a python callable object from a clojure function. This is called for you
if you use as-python
on an implementation of IFn.
Options:
:arg-converter
- Function called for each function argument before your ifn gets access to it. Defaults to->jvm
.:result-converter
- Function called on return value before it gets returned to python. Must return a python object. Defaults to->python
; the result will get an extra incref before being returned to Python to account for the implied tracking ofas-python
or->python
.:name
- Name of the python method. This will appear in stack traces.:doc
- documentation for method.
make-fastcallable
(make-fastcallable item)
Wrap a python callable such that calling it in a tight loop with purely positional arguments is a bit (2x-3x) faster.
Example:
user> (def test-fn (-> (py/run-simple-string "def spread(bid,ask):
return bid-ask
")
(get :globals)
(get "spread")))
#'user/test-fn
user> test-fn
<function spread at 0x7f330c046040>
user> (py/with-gil (time (dotimes [iter 10000]
(test-fn 1 2))))
"Elapsed time: 85.140418 msecs"
nil
user> (py/with-gil (time (dotimes [iter 10000]
(test-fn 1 2))))
"Elapsed time: 70.894275 msecs"
nil
user> (with-open [test-fn (py/make-fastcallable test-fn)]
(py/with-gil (time (dotimes [iter 10000]
(test-fn 1 2)))))
"Elapsed time: 39.442622 msecs"
nil
user> (with-open [test-fn (py/make-fastcallable test-fn)]
(py/with-gil (time (dotimes [iter 10000]
(test-fn 1 2)))))
"Elapsed time: 35.492965 msecs"
nil
make-instance-fn
(make-instance-fn ifn options)
(make-instance-fn ifn)
Make an callable instance function - a function which will be passed the 'this'
object as it's first argument. In addition, this function calls make-callable
with a arg-converter
defaulted to as-jvm
. See documentation for
make-callable and libpython-clj2.python.class/make-tuple-instance-fn.
make-kw-instance-fn
(make-kw-instance-fn ifn options)
(make-kw-instance-fn ifn)
Make an kw callable instance function - function by default is passed 2 arguments, the positional argument vector and a map of keyword arguments. Results are marshalled back to python using libpython-clj2.python.fn/bridged-fn-arg->python which is also used when bridging an object into python. See documentation for make-callable libpython-clj2.python.class/make-kw-instance-fn.
path->py-obj
(path->py-obj item-path & {:keys [reload?]})
Given a string such as "builtins" or "builtins.list", load the module or the class object in the module.
Options:
:reload
- Reload the module.
py*
macro
(py* x method args)
(py* x method args kwargs)
Special syntax for passing along *args and **kwargs style arguments to methods.
Usage:
(py* obj method args kwargs)
Example:
(def d (python/dict)) d ;;=> {} (def iterable [:a 1 :b 2]) (def kwargs {:cat "dog" :name "taco"}) (py* d update iterable kwargs) d ;;=> {"a": 1, "b": 2, "cat": "dog", "name": "taco"}
py**
macro
(py** x method kwargs)
(py** x method arg & args)
Like py*, but it is assumed that the LAST argument is kwargs.
py.
macro
(py. x method-name & args)
Class/object method syntax. (py. obj method arg1 arg2 ... argN) is equivalent to Python's obj.method(arg1, arg2, ..., argN) syntax.
py.-
macro
(py.- x arg)
Class/object getter syntax. (py.- obj attr) is equivalent to Python's obj.attr syntax.
py..
macro
(py.. x & args)
Extended accessor notation, similar to the ..
macro in Clojure.
(require-python 'sys) (py.. sys -path (append "/home/user/bin"))
is equivalent to Python's
import sys sys.path.append('/home/user/bin')
SPECIAL SYNTAX for programmatic *args and **kwargs
Special syntax is provided to meet the needs required by Python's *args and **kwargs syntax programmatically.
(= (py.. obj (method args)) (py obj methods args))
(= (py.. obj (method args kwargs)) (py obj method args kwargs))
(= (py.. obj (method kwargs)) (py obj method kwargs))
(= (py.. obj (method arg1 arg2 arg3 ... argN kwargs)) (py obj method arg1 arg2 arg3 ... argN kwargs) (py* obj method arg1 arg2 arg3 ... argN kwargs))
These forms exist for when you need to pass in a map of options in the same way you would use the f(*args, **kwargs) forms in Python.
run-simple-string
(run-simple-string program & {:keys [globals locals]})
Run a string expression returning a map of {:globals :locals}. This uses the global main dict under the covers so it matches the behavior of the cpython implementation with the exception of returning the various maps used.
Note this will never return the result of the expression: https://mail.python.org/pipermail/python-list/1999-April/018011.html
Globals, locals may be provided but are not necessary.
Implemented in cpython as:
PyObject *m, *d, *v; m = PyImport_AddModule("main"); if (m == NULL) return -1; d = PyModule_GetDict(m); v = PyRun_StringFlags(command, Py_file_input, d, d, flags); if (v == NULL) { PyErr_Print(); return -1; } Py_DECREF(v); return 0;
set-attr!
(set-attr! pyobj attname attval)
Set an attribute on a python object. Returns pyobj.
set-item!
(set-item! pyobj item-name item-val)
Set an item on a python object using setitem
stack-resource-context
macro
(stack-resource-context & body)
Create a stack-based resource context. All python objects allocated within this context will be released at the termination of this context. !!This means that no python objects can escape from this context!! You must use copy semantics (->jvm) for anything escaping this context. Furthermore, if you are returning generic python objects you may need to call (into {}) or something like that just to ensure that absolutely everything is copied into the jvm.
with
macro
(with bind-vec & body)
Support for the 'with' statement in python: (py/with item (py/call-attr testcode-module "WithObjClass" true fn-list) (py/call-attr item "doit_err"))
with-gil
macro
(with-gil & body)
Capture the gil for an extended amount of time. This can greatly speed up operations as the mutex is captured and held once as opposed to fine grained grabbing/releasing of the mutex.
with-gil-stack-rc-context
macro
(with-gil-stack-rc-context & body)
Capture the gil, open a resource context. The resource context is released
before the gil is leading to much faster resource collection. See documentation
on stack-resource-context
for multiple warnings; the most important one being
that if a python object escapes this context your program will eventually, at
some undefined point in the future crash. That being said, this is the recommended
pathway to use in production contexts where you want defined behavior and timings
related to use of python.
with-manual-gil
macro
(with-manual-gil & body)
When running with -Dlibpython_clj.manual_gil=true, you need to wrap all accesses to the python runtime with this locker. This includes calls to require-python or any other pathways.
(with-manual-gil
...)
with-manual-gil-stack-rc-context
macro
(with-manual-gil-stack-rc-context & body)
When running with -Dlibpython_clj.manual_gil=true, you need to wrap all accesses to the python runtime with this locker. This includes calls to require-python or any other pathways. This macro furthermore defines a stack-based gc context to immediately release objects when the stack frame exits.