Lisp Pimpin'

Started by
387 comments, last by Mercury 18 years, 8 months ago
Quote:Original post by CoffeeMug
Quote:Original post by Tron3k
Firstly, it is related in that I don't want to use two languages at once. It is too much trouble.

If this is true it's a huge weakness of Lisp implementations (and therefore Common Lisp standard itself for all practical purposes) because in the modern world very few serious systems are written in a single language. If it isn't true then this point is a non-issue.
What I meant is that it's too much trouble for me. I don't want to even think about low-level issues like interfacing between two languages; I want to focus on the parts I find fun.

I don't know anything about Erlang, but supposing that it exports a C interface from a DLL, it wouldn't be hard to interact with it. You could then write a wrapper over that C interface in Lisp, which would make the interface between Lisp and Erlang as seamless as possible.
“[The clergy] believe that any portion of power confided to me, will be exerted in opposition to their schemes. And they believe rightly: for I have sworn upon the altar of God, eternal hostility against every form of tyranny over the mind of man” - Thomas Jefferson
Advertisement
Quote:Original post by xhyldazhk
The fact is, joining C/C++ code with LISP code is more difficult than joining C/C++ code with Python code.

People keep telling how easy it's to mix C & Python. Why is that?

In Python, AFAIK, you need to write wrappers for even basic c-functions. In CMUCL (and most of other lisps I know) you just load the library, define functions and call them. It's even simpler than doing the same thing in C. For libraries with simple api's - Open Dynamics Engine, for example - it's easy to automatically generate the necessary foreign declaritions.
There are two major complications in using c/c++ code from lisp that I have run into and I'd like to know how Python handles them.
- Interfacing with C++ can be a pain. You need to write wrappers and there's no standard way to extend C++ classes or call their methods, not to mention templates and whatever. Does Python's foreign-api handle these or is it upto SWIG. If SWIG does that how robust is it? If it works it can be done for CL too. ISTR that there is an (experimental?) clisp module in SWIG already and the Fetter project in common-lisp.net is aiming for portable c/c++ wrapper generator for CL.
- C likes to be on top. Most c/c++ code is not designed to be used from languages like CL (surprisingly). Some may grab the command loop, screw up stack when errors happen and control jumps from c to lisp and other hairy things that go beyond my understanding. So at worst it almost like coding in C :-(. But many libraries work without a glitch, OpenGL, SDL, ODE, FLTK, Irrlicht(maybe) etc.

What gamedevelopment related Python libs would you like to see in CL? Are there mature 3d engine bindings for Python, maybe they could be ported to CL. I have only been following CL for ~2 years but the number of libraries is constantly growing.
---from www.yellow-hut.com/blog
Quote:Original post by Nathan Baum
Using two languages at once is generally a lot of trouble with any two languages where the underlying data and control models are vastly different.

People tend to expect complex things to be complex but simple things can reasonably be expected to be simple. At this point in time there is absolutely no technical reason I can think of that prevents a Lisp implementation from calling C functions in a friendly manner.
(import-c-library "blah.h" "blah.lib")(some-function-in-blah 1 2)

Naturally the implementation must deal with calling conventions and data marshalling but once again, there is no reason why this should be complex for simple things. Intelligent defaults work.
Quote:Original post by Nathan Baum
Lisp's saving grace in this regard is that if you can't use another language, you can at least spend some time making a DSL which is similarly well-suited to the problem domain. For example, you could make a concurrent and distributed DSL which is comparable to Erlang. Doing the same in a less extensible language would be impracticable.

If you familiarize yourself with Erlang you'll find that the bulk of the features that make it an attractive language for its problem domain either cannot be transferred to Lisp all together or aren't helped by Lisp DSL creation mechanisms. In particular, there is no practical way to transfer Erlang's process scheduling system to Lisp. Additionally the bulk of Erlang's code lies in an efficient networking code implementation. DSLs in Lisp are nice but I'd have to spend years implementing a system that would probably be crappier than a current Erlang implementation. In this case rolling my own implementation is really not an option.
Quote:Original post by Nathan Baum
Only languages for which compiled programs include rich typing information are likely to be at all easily to load into Lisp at runtime.

One could trivially parse a C header file to get the same information. The question is implementing it. Lisp is a tremendously extensible language but "roll your own" is not an answer. If I'm writing a MMORPG or a trading system I cannot afford to do all these things. I need them to come out of the box.
Quote:Original post by CoffeeMug
Quote:Original post by Nathan Baum
Only languages for which compiled programs include rich typing information are likely to be at all easily to load into Lisp at runtime.

One could trivially parse a C header file to get the same information.


No, not at all. By the standards of modern FPLs, C APIs are often not type safe. You simply cannot extract the necessary information from the entire C program - it isn't there.

The IDL partially addresses this by supplementing C header files with declarations stating whether variables are going in or coming out. But even IDL doesn't provide enough information.

I've heard that Haskell has an excellent FFI. I've used OCaml's a bit and it isn't great. I'd expect Lisp to be somewhere between the two.
Quote:Original post by jdh30
No, not at all. By the standards of modern FPLs, C APIs are often not type safe.

Who cares? When I interop with C code I surrender my safety expectations. Most C functions that deal with structs and primitive types are trivial to interop with. If a structure or a function has a reference to an obscure typeless array, well, that's my problem. Perl has excellent facilities to format its datatypes into binary, why can't Lisp? If there's a void* then the programmer automatically assumes responsibility:
; marshall formats integer 5 into a four byte; little endian array that carries over upon overflow(native-c-function (marshall 5 "4lc"))
Quote:Original post by CoffeeMug
Quote:Original post by Nathan Baum
Using two languages at once is generally a lot of trouble with any two languages where the underlying data and control models are vastly different.

People tend to expect complex things to be complex but simple things can reasonably be expected to be simple. At this point in time there is absolutely no technical reason I can think of that prevents a Lisp implementation from calling C functions in a friendly manner.
(import-c-library "blah.h" "blah.lib")(some-function-in-blah 1 2)

Naturally the implementation must deal with calling conventions and data marshalling but once again, there is no reason why this should be complex for simple things. Intelligent defaults work.

This would work.

As jdh30 pointed out (in typical alarmist form [wink]), there are still incompatabilities to look out for.

If you ever passed a pointer to an object to an imported function, you wouldn't know if it was safe for the GC to collect it. Either the object is made permanent; or you have to provide custom annotation to indicate that the called function won't be aliasing the value.

For example, we know that "glTexImage2D" doesn't alias its "pixels" argument, but nothing in the declaration tells Lisp that this is the case: it couldn't ever be sure it was safe to free the pixel array. You'd have to explicitly tell Lisp to free it.

It may be worth noting that C99's "restrict" qualifier is of no use here, in two ways.

The first way is that the aliasing restriction of a restrict-qualified pointer only applies between other restrict-qualified pointers in scope, and not to non-restrict-qualified pointers in scope. For example, the standard mandates that "strtok" alias its first argument, which the standard mandates is restrict-qualified.

The second way is that the language doesn't seek to enforce the restriction, likely because it is defined in such a way that enforcing it would be prohibitively expensive. So even if "restrict" did have useful semantics, Lisp couldn't be sure that the programmer had obeyed them.
Quote:
If you familiarize yourself with Erlang you'll find that the bulk of the features that make it an attractive language for its problem domain either cannot be transferred to Lisp all together or aren't helped by Lisp DSL creation mechanisms. In particular, there is no practical way to transfer Erlang's process scheduling system to Lisp. Additionally the bulk of Erlang's code lies in an efficient networking code implementation. DSLs in Lisp are nice but I'd have to spend years implementing a system that would probably be crappier than a current Erlang implementation. In this case rolling my own implementation is really not an option.

In terms of the simple facts, Lisp is Turing complete and therefore can perform any computation Erlang can perform.

Issues of practicality are fair game, though. To make an Erlisp that is as efficient as Erlang would either mean using a Lisp implementation which itself featured support for efficient distributed and parallelised processing (there are Lisps that run on parallelising supercomputers, but I don't know if they're generally available) or effectively making a complete Erlang implementation.

Whether or not it is acceptable to make your own implementation of Erlang or to use a distributed and parallelised DSL which is 'like' Erlang but not an exact clone depends upon your needs.

If you want to make a direct clone of Erlang's semantics with Lisp's syntax, the easiest implementation technique would likely be to write a function which translated your Erlisp DSL into Erlang, gave that to the Erlang compiler, and then used FFI to import it back in.

On the other hand, if you just want something that can evaluate expressions in parallel on different hosts, and a simple message-passing control system is sufficient for your needs, then a DSL may be a reasonable choice when compared to linking Lisp to Erlang.

If the non-distributed and non-parallelised aspects of the problem aren't susceptible to simplification via Lisp's features, then you might as well use Erlang for the whole thing.
Quote:Original post by CoffeeMug
Perl has excellent facilities to format its datatypes into binary, why can't Lisp?

There already are implementation specific ways to do these kinds of things in CL. Or do you mean it should be in the standard? In that case it's the old standard vs. implentation specified languages debate. Are there any languages which define in their standard a way to interact with C (except maybe C++ and C itself)?
---from www.yellow-hut.com/blog
Quote:Original post by Nathan Baum
As jdh30 pointed out (in typical alarmist form [wink]), there are still incompatabilities to look out for.

There are a few solutions to them. Anyway, that's not the point. The point is that there are no compelling reasons against having interop services for Lisp out of the box. To the contrary, there are plenty of reasons in favor.
Quote:Original post by Nathan Baum
In terms of the simple facts, Lisp is Turing complete and therefore can perform any computation Erlang can perform.

This is really a useless statement [smile] Obviously I could implement Erlang in Lisp. What I said is that some Erlang features cannot be transferred to Lisp.
Quote:Original post by Nathan Baum
Issues of practicality are fair game, though.

If I'm in the business of writing trading systems I'm only concerned with writing trading systems. I really can't afford to rewrite technologies that already exist and do things much better than I could with my resources.
Quote:Original post by Flatlander
Or do you mean it should be in the standard? In that case it's the old standard vs. implentation specified languages debate. Are there any languages which define in their standard a way to interact with C (except maybe C++ and C itself)?

That's exactly what I mean. Having implementation specific features isn't good enough for the real world. Saying "one could implement this in Lisp in a really elegant manner" isn't good enough for the real world. It's fairly interesting that the languages much of the industry has switched/is switching to (Java, C#, Python) do have interop with C in the standard.
Quote:Original post by CoffeeMug
Quote:Original post by Flatlander
standard vs. _implentation_specified_languages_

That's exactly what I mean. Having implementation specific features isn't good enough for the real world. Saying "one could implement this in Lisp in a really elegant manner" isn't good enough for the real world. It's fairly interesting that the languages much of the industry has switched/is switching to (Java, C#, Python) do have interop with C in the standard.

I don't know about Java & C# but I don't think it's correct to say that Python has a standard way to interact with C. Or do the few Python implementations have a common foreign-api? Does Python have a standard? The CPython FFI is no more standard than, for example, CMUCL's. Other than that I somewhat agree, and there already are projects to define an unified FFI-api for CL's. I'm not sure if something like that belongs in the standard in the first place... With what language and at what level should the co-operation be defined? If ANSI-CL had defined C<->Lisp interface now people would be crying for C++/C#/whatever standarization.
---from www.yellow-hut.com/blog

This topic is closed to new replies.

Advertisement