idapython-src/docs/howto/swig.md

This is an FAQ-style repository of SWiG incantations that are regularly needed.

Remove an argument from a prototype (because it doesn't make sense in the context of IDAPython)

%typemap(in,numinputs=0) int length
{
    $1 = -1; // always -1 in IDAPython
}

Add a parameter to a C++ notification

Assume a notification named struc_renamed, to which you just added a bool success parameter (to carry along information about the operation's success so far):

struc_renamed,          ///< A structure type has been renamed.
                        ///< \param sptr (::struc_t *)
                        ///< \param success (bool)  // <--------- new

That new parameter will be picked up by the hooks-producing code automatically (great!), which means existing hooks are now broken (bad.)

We need to add support for calling into "old-style" hooks. That's done through the patch_codegen.py mechanism, and in particular the director_method_call_arity_cap rule. Something like this should do:

"SwigDirector_IDB_Hooks::struc_renamed" : [
    ("director_method_call_arity_cap", (
        False, # add GIL lock
        "struc_renamed",
        "(method ,(PyObject *)obj0,(__argcnt < 3 ? nullptr : (PyObject *)obj1), nullptr)",
        "(swig_get_self(), (PyObject *) swig_method_name ,(PyObject *)obj0,(__argcnt < 3 ? nullptr : (PyObject *)obj1), nullptr)")
    )),
],

Handle a virtual method's "output" parameter in a director

Assume you have a virtual method taking an 'output' argument:

void merge_node_helper_t::get_column_headers(qstrvec_t *out, ...)

The neat thing to do here is to let Python implementations do the following:

def get_column_headers(self, ...):
    # ...
    return ["Name", "Address"]

To achieve that, you want to use the directorargout typemap:

%typemap(directorargout) qstrvec_t * (qstrvec_t tmp)
{ // %typemap(directorargout) qstrvec_t *
  if ( PyW_PySeqToStrVec(&tmp, $result) >= 0 )
  {
    $1->swap(tmp);
  }
  else
  {
    Swig::DirectorTypeMismatchException::raise(
            SWIG_ErrorType(SWIG_TypeError),
            "in output value of type 'qstrvec_t' in method '$symname'");
  }
}

"intercept" access to a structure/class field, in order to perform extra work

For example, idainfo.lflags bits should be set using proper setters, because they can have side-effects.

. tell swig to consider the member as unreachable:

%ignore idainfo::lflags;

. extend the type to "manually" provide the field:

%extend idainfo
{
  // ...
  uint32 _get_lflags() const { return $self->lflags; }
  void _set_lflags(uint32 _f)
  {
    // do the job here
  }

  %pythoncode {
    lflags = property(_get_lflags, _set_lflags)