allow red functions to be passed as arguments and return values by seibert · Pull Request #608 · spylang/spy

seibert · 2026-06-25T21:43:17Z

This is a redo of #607 based on out-of-band discussion with @antocuni. In this version, we don't need a new type for C callbacks, we use the existing W_FuncType, and make the minimum changes required to allow red functions to be passed around as arguments. The end goal is the same: to enable external modules to accept SPy functions as C callbacks. Calling a function passed as an argument is still not allowed in this PR.

This PR adds a builtin @functype decorator to make it easier to define a new function type, so you can write stuff like:

@functype
def CB(x: i32) -> i32:
    pass

@blue
def make_adder(n: i32) -> CB:
    def adder(x: i32) -> i32:
        return x + n
    return adder

(This PR was created with AI assistance.)

… the C backend

JeffersGlass · 2026-06-26T13:42:27Z

@functype essentially allows up to not have to write a body for the function, yeah? I've always just used type(CB) in place or assigned to a variable:

def CB(x: i32) -> i32:
    return 0

@blue
def make_adder(n: i32) -> type(CB):
    def adder(x: i32) -> i32:
        return x + n
    return adder

seibert · 2026-06-26T16:38:10Z

Talking with @antocuni, it is likely that @functype works on accident right now (the pass in the body of CB is only working because it is a blue function and the signature isn't being checked), so we might want a different approach.

antocuni

it's mostly good, but with a lot of nitpicks here and there that overall degrade code quality and should be fixed.

The biggest thing to solve is the change to SPyVM.make_fqn_const which is plainly wrong, see the inline comment.

antocuni · 2026-06-29T11:39:05Z

+                    if d != fqn and d not in seen:
+                        seen.add(d)
+                        deps.append(d)
+            return deps


I think this deserves its own unit test.
See e.g.:

spy/spy/tests/test_backend_c.py

Lines 111 to 140 in 0f2192b

def test_struct_deps(self, vm):

src = """

from unsafe import gc_ptr

@struct

class Color:

name: str

@struct

class Point:

x: i32

y: i32

@struct

class Rect:

a: Point

b: Point

color: gc_ptr[Color]

"""

backend = self.compile_until_CBackend(vm, src)

def deps(fqn_str: str) -> list[str]:

return [str(d) for d in backend.get_type_deps(FQN(fqn_str))]

# A struct containing only primitive/pointer fields has no by-value

# type deps. Pointers do NOT count as dependencies, since we already

# emit a forward declaration for the pointee type.

assert deps("test::Color") == []

assert deps("test::Point") == []

assert deps("test::Rect") == ["test::Point"]

antocuni · 2026-06-29T11:53:42Z

+            c_params = ", ".join(str(self.ctx.w2c(p.w_T)) for p in w_ft.params)
+        else:
+            c_params = "void"
+        self.tbh_fwdecl.wl(f"typedef {c_ret} (*{c_name})({c_params});")


this works and it's good enough for this PR.
But note that it produces an incredibly ugly C type name.

This happens because function types get an FQN which depends on their "structural" shape. For example for a functype def(i32, i32) -> f64 the fqn is:

FQN('builtins::def[i32, i32, f64]')

And the C typedef is:

typedef double (*spy_builtins$def__builtins$i32_builtins$i32_builtins$f64)(int32_t, int32_t);

It's unclear how to improve it though (apart making builtins shorter, which maybe it's a good idea). The nice property of this is that if you compile two spy programs separately (e.g. a main executable and a lib) their function types are automatically compatible at the C level.

Random ideas for how to improve it:

give a unique numering and call them FuncType1, FuncType2, etc. They are more opaque, but complex function types will have unreadable names anyway. The problem is that separate compilation becomes harder because you need to agree on numbering.

do it at the spy level. Declare e.g. that @functype defines a NEW type with its own fqn:

@functype def BinOp(x: int, y: int) -> int: pass

Here, we would define a fresh functype whose fqn is mod::BinOp: the advantage is that the redshift output and the C output become very clear. The new type would have conversion functions to convert to and from def(int, int) -> int.

I kind of like option 2, but my only doubt is that most PLs do NOT treat function types as new types. The only exception seems to be go, from what I understand.

antocuni · 2026-06-29T11:55:57Z

+# red def(i32) -> i32: a callback that takes one i32 and returns i32
+_w_cb_type = W_FuncType.new([FuncParam(B.w_i32, "simple")], B.w_i32)
+CB = Annotated[W_Func, _w_cb_type]
+


this seems the wrong place where to put this test code, as it's not about out-of-tree at all.
I'd put it in vm/modules/_testing_helpers.py

antocuni · 2026-06-29T11:57:41Z

        """)
        assert mod.foo() == "hello from mymod"
+
+    def test_c_callback(self):


and these tests should go to test_functype.py

antocuni · 2026-06-29T12:00:35Z

+        assert w_CB.w_restype is B.w_i32
+        assert [p.w_T for p in w_CB.params] == [B.w_i32, B.w_i32]
+        assert w_CB.color == "red"
+        assert w_CB.kind == "plain"


if you want to simplify this test, you can probably just assert w_CB.name == 'def (i32, i32) -> i32'.

antocuni · 2026-06-29T12:04:36Z

+    def test_functype_at_call_site(self):
+        mod = self.compile("""
+        @functype
+        def CB(x: i32, y: i32) -> i32:
+            pass
+
+        def apply(cb: CB, x: i32, y: i32) -> i32:
+            return x + y
+
+        def my_add(a: i32, b: i32) -> i32:
+            return a + b
+
+        def run() -> i32:
+            return apply(my_add, 3, 4)
+        """)
+        assert mod.run() == 7


and this seems to be the exact same test as above?

antocuni · 2026-06-29T12:06:58Z

+    def test_signature_mismatch_wrong_ret(self):
+        src = """
+        @functype
+        def CB(x: i32) -> bool:
+            pass
+
+        def my_fn(x: i32) -> i32:
+            return x
+
+        def bad() -> CB:
+            return my_fn
+        """
+        errors = expect_errors("mismatched types")
+        self.compile_raises(src, "bad", errors)
+
+    def test_signature_mismatch_wrong_argcount(self):
+        src = """
+        @functype
+        def CB(x: i32, y: i32) -> i32:
+            pass
+
+        def my_fn(x: i32) -> i32:
+            return x
+
+        def bad() -> CB:
+            return my_fn
+        """
+        errors = expect_errors("mismatched types")
+        self.compile_raises(src, "bad", errors)
+
+    def test_signature_mismatch_wrong_arg_type(self):
+        src = """
+        @functype
+        def CB(x: f64) -> i32:
+            pass
+
+        def my_fn(x: i32) -> i32:
+            return x
+
+        def bad() -> CB:
+            return my_fn
+        """
+        errors = expect_errors("mismatched types")
+        self.compile_raises(src, "bad", errors)


these have nothing to do with func types.
Typechecking function calls is already tested elsewhere.

Kill.

antocuni · 2026-06-29T12:08:39Z

+    def test_blue_func_rejected(self):
+        # A @blue function's functype has a different FQN than a red functype,
+        # so passing it where a @functype type is expected fails with a type mismatch.
+        src = """
+        @functype
+        def CB(x: i32) -> i32:
+            pass
+
+        @blue
+        def my_fn(x: i32) -> i32:
+            return x
+
+        def get_cb() -> CB:
+            return my_fn
+        """
+        errors = expect_errors("mismatched types")
+        self.compile_raises(src, "get_cb", errors)


this is an interesting case, we can keep it.
But expect_errors should be MORE precise and check that we get a reasonable explanation/error annotation.

I have noticed multiple times that claude tends to be very bad at writing these tests.

antocuni · 2026-06-29T12:09:01Z

+    def test_blue_factory(self):
+        # A @blue function can generate a red callback that captures compile-time
+        # constants. After redshifting, captures are inlined so each becomes a
+        # standalone C symbol.
+        self.compile("""
+        @functype
+        def CB(x: i32) -> i32:
+            pass
+
+        @blue
+        def make_adder(n: i32) -> CB:
+            def adder(x: i32) -> i32:
+                return x + n
+            return adder
+
+        add5: CB = make_adder(5)
+        add10: CB = make_adder(10)
+        """)


kill, we already have an end-to-end test.

antocuni · 2026-06-29T12:25:10Z

+            if fqn in self.globals_w:
+                return fqn


this is VERY wrong and it's a code smell. Bad claude 😡.
Early in this function we do a reverse lookup:

fqn = self.reverse_lookup_global(w_val) if fqn is not None: return fqn

so, if w_val is ALREADY in the globals, we returned the FQN.
If we reach this stage, it means that we need to add w_val to the globals, and the assert is a sanity check to ensure that for some reason the FQN doesn't already exist.
It turns out that the FQN builtins::functype is already taken by W_FuncType (see
function.py):

W_FuncType._w = W_Type.declare(FQN("builtins::functype"))

So the failing assert here was catching a real bug. This means that we should probably rename the @functype decorator.

antocuni · 2026-06-29T12:35:48Z

@functype essentially allows up to not have to write a body for the function, yeah? I've always just used type(CB) in place or assigned to a variable:

@JeffersGlass type(CB) works but I think it would be good to have a way to give explicit names to function types, that's why I suggested the @functype decorator.

Talking with @antocuni, it is likely that @functype works on accident right now (the pass in the body of CB is only working because it is a blue function and the signature isn't being checked), so we might want a different approach.

Yes, I think that now it works by chance, but we could make it working with ...:

@functype
def CB(x: int, y: int) -> float: ...

Speaking of which, I'm not 100% sure about the name @functype. What about:

@def_functype
def CB(x: i32, y: i32) -> i32: ...

Other possibilities:

@define_functype
@functype_alias
@make_functype
@get_functype
@new_functype

JeffersGlass · 2026-06-29T12:55:04Z

My personal favorites among the names you listed @antocuni are def_functype and make_functype, but that's only based on vibes 😄

seibert mentioned this pull request Jun 25, 2026

Create a CFuncPtr type and allow red functions to be passed as callbacks to C libraries #607

Closed

seibert force-pushed the functype branch 2 times, most recently from 75628fc to 917c74e Compare June 25, 2026 22:26

allow functypes to be used as argument and return type and emitted by…

25746a6

… the C backend

seibert force-pushed the functype branch from 917c74e to 25746a6 Compare June 25, 2026 23:48

antocuni requested changes Jun 29, 2026

View reviewed changes

	def test_struct_deps(self, vm):
	src = """
	from unsafe import gc_ptr

	@struct
	class Color:
	name: str

	@struct
	class Point:
	x: i32
	y: i32

	@struct
	class Rect:
	a: Point
	b: Point
	color: gc_ptr[Color]
	"""
	backend = self.compile_until_CBackend(vm, src)

	def deps(fqn_str: str) -> list[str]:
	return [str(d) for d in backend.get_type_deps(FQN(fqn_str))]

	# A struct containing only primitive/pointer fields has no by-value
	# type deps. Pointers do NOT count as dependencies, since we already
	# emit a forward declaration for the pointee type.
	assert deps("test::Color") == []
	assert deps("test::Point") == []
	assert deps("test::Rect") == ["test::Point"]

Uh oh!

Conversation

seibert commented Jun 25, 2026

Uh oh!

JeffersGlass commented Jun 26, 2026

Uh oh!

seibert commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antocuni left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

antocuni commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JeffersGlass commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

seibert commented Jun 26, 2026 •

edited

Loading

antocuni commented Jun 29, 2026 •

edited

Loading

JeffersGlass commented Jun 29, 2026 •

edited

Loading