Files
atlas0fd00m 4979ea079f Vtrace conversion and bugfixes (#406)
* lotsa work making it all work, with segments and maps and collapsing adjacent maps and bugfixes and...

* pagemap and memorymap deletion

* update unittests

* memory-related updates (and unittests for delMemoryMap)

* unittests and MapNotFoundException

* update unittest to test appropriately

* change in symstore fix

* more bugfixes for vdb/win32.  symbols are a bit of a mess and may need to be refactored in the future.

* make syms work (do_syms and do_bp) on Win32

* unittest mods per @rakuy0

* aaaaand this will make unittests break.
unittest discovered a bug in symstore about storing subresolvers... still need to fix the bug.

* lockstep emulator class and some improvements for Win32 PEB/TEB capture

* enhancements for envitools.LockstepEmulator class

* cleanup and import bugfix

* collapseMemoryMaps::strict

* bugfix:  deleting symbols and tests

* bugfix: Win32 detaching

* updates per @rakuy0

* touchups per @rakuy0

* tweaking the test to account for different versions of Linux and python

* updates per @rakuy0 and improvements from i386_emu_... PR.  we're housing the refugee code here while the cull takes place in PR#405

* update to identify FreeBSD ELF files (OS matters on some of these vtrace and emu changes)

* a few bugfixes for Elf and Vtrace handling of Elfs.

* Break On Library Init (and stub for Library Load)
and bugfix for clicking EFLAGS gui buttons.

* modified flaky unittest

* removed prints

* bugfix for event/threading for notify/breakpoints

* Finally!  LockStepper Class moving into vtrace.envitools.  Raw move here... updates to follow (for easy diffing between commits)

* bugfix

* revamped the LockStepper class (still have to remove LockStepEmulator class and revamp "lockStepEmulator" when we're done)

* oops

* bugfix

* change a few INS_SYSTEM x86/x64 instruction opcodes to be unique

* unify @rakuy0's and my lockstep emu classes and clean up

* de-x86ify

* bsd commented code removal
.gitignore to ignore docs build files

* bugfix: vwFromTrace() call to addFile was handing in the md5 object, not a serializable string (hexdigest)
bugfix: vdb snapshot wasn't capturing TEB's from a trace object, vw/emuFromTrace wasn't allowing for PEB/TEBs tracking from snapshot

* bugfix: str versus bytes

* cleanup 'Comparing' print/log message

* updates per @rakuy0

* finish the docstr (per rakuy0)

* finish the docstr (per rakuy0) (for realz this time, last commit was actually the @idlethread change)

* document user interface for InteractiveLSMon

* MM_* removed from envi.memory and left only in envi.const

* allow --LL and --LI settings to persist between different traces in the same session (instead of only for the first run and only if provided "-c /path/to/proggy" at the command line.  in the future, make this more cohesive by setting vdbbin to create the `db` and use `db.newTrace()` instead of punching low-level into vtrace.getTrace.  this will have to include handling of `platargs` which is beyond the scope of this PR currently and would unnecessarily delay review/merging

* cleanup per rakuy0

* cleanups per rakuy0

* bugfix (unittests have been screaming about a typo) and minor cleanup.

* kwargs to get new trace able to handle platform magic.
logging of --LI and --LL effects.

* yes, @rakuy0, that should do something ;)

* windows debugging privs (tested on Win7 and Win10)
debugging print statements that need to be cut/converted to logs

* Break on Library Load/Init bugfixes

* make POSIX (Linux) catch Library Loads and raise the correct notifiers.
this requires hooking a function in `ld` which causes libraries to be rediscovered and unresolved breakpoints to be resolved if possible.

* outdated code causing Python warnings.

* decouping Vtrace from VDB, where only the latter has a trace.db.config (or trace.db, for that matter)

* no prints!

* equaling out the unresolved breakpoint warnings.

* update vtrace unittest

* remove need for pywin32 (in README)

* remove dependency on pywin32 and winadmin

* vprint on both VdbTrace and Trace objects.
and string/bytes bugfixes.

* add to the BP execution context

* don't need to hand in **kwargs to `self.getTrace()` in fact, that's bad.

* REMOVE collapseMaps functionality.  no longer important since we can emulate/read across memory maps now.
there are many other bugfixes in the branch that should be merged, and this was always a little wonky.

* lost an import somewhere.

* test_privs was getting caught by unittests :)

* remove tests for the collapseMaps that i just yanked out of this PR.

* update checkprivs to use only ctypes instead of pywin32.
TODO: wrap this ability into indicating that Windows users don't have sufficient rights for debug.

* updates per @rakuy0

---------

Co-authored-by: James Gross <45212823+rakuy0@users.noreply.github.com>
2023-05-01 12:06:32 -04:00

182 lines
6.8 KiB
Python

"""
Vtrace notitifers base classes and examples
Vtrace supports the idea of callback notifiers which
get called whenever particular events occur in the target
process. Notifiers may be registered to recieve a callback
on any of the vtrace.NOTIFY_FOO events from vtrace. One notifier
*may* be registered with more than one trace, as the "notify"
method is passed a reference to the trace for which an event
has occured...
"""
# Copyright (C) 2007 Invisigoth - See LICENSE file for details
import logging
import traceback
import vtrace
logger = logging.getLogger(__name__)
class Notifier(object):
"""
The top level example notifier... Anything which registers
itself for trace events or tracegroup events should implement
the notify method as shown here.
"""
def __init__(self):
"""
All extenders *must* call this. Mostly because all the
goop necessary for the remote debugging stuff...
(if notifier is instantiated on server, all is well, if it's
on the client it needs a proxy...)
"""
pass
def handleEvent(self, event, trace):
"""
An "internal" handler so if we need to do something
from an API perspective before calling the notify method
we can have a good "all at once" hook
"""
self.notify(event, trace)
def notify(self, event, trace):
logger.info("Got event: %d from pid %d", event, trace.getPid())
class VerboseNotifier(Notifier):
def notify(self, event, trace):
logger.info("PID %d - ThreadID (%d) got", trace.getPid(), trace.getMeta("ThreadId"))
if event == vtrace.NOTIFY_ALL:
("WTF, how did we get a vtrace.NOTIFY_ALL event?!?!")
elif event == vtrace.NOTIFY_SIGNAL:
signo = trace.getCurrentSignal()
("vtrace.NOTIFY_SIGNAL %d (0x%08x)" % (signo, signo))
if trace.getMeta("Platform") == "windows":
logger.info(repr(trace.getMeta("Win32Event")))
elif event == vtrace.NOTIFY_BREAK:
logger.info("vtrace.NOTIFY_BREAK")
logger.info("\tIP: 0x%08x", trace.getProgramCounter())
elif event == vtrace.NOTIFY_SYSCALL:
logger.info("vtrace.NOTIFY_SYSCALL")
elif event == vtrace.NOTIFY_CONTINUE:
logger.info("vtrace.NOTIFY_CONTINUE")
elif event == vtrace.NOTIFY_EXIT:
logger.info("vtrace.NOTIFY_EXIT")
logger.info("\tExitCode: %d", trace.getMeta("ExitCode"))
elif event == vtrace.NOTIFY_ATTACH:
logger.info("vtrace.NOTIFY_ATTACH")
elif event == vtrace.NOTIFY_DETACH:
logger.info("vtrace.NOTIFY_DETACH")
elif event == vtrace.NOTIFY_LOAD_LIBRARY:
logger.info("vtrace.NOTIFY_LOAD_LIBRARY")
logger.info("\tLoaded library %s", trace.getMeta('LatestLibrary'))
elif event == vtrace.NOTIFY_UNLOAD_LIBRARY:
logger.info("vtrace.NOTIFY_UNLOAD_LIBRARY")
elif event == vtrace.NOTIFY_CREATE_THREAD:
logger.info("vtrace.NOTIFY_CREATE_THREAD")
logger.info("\tNew thread - ThreadID: %d", trace.getMeta("ThreadId"))
elif event == vtrace.NOTIFY_EXIT_THREAD:
logger.info("vtrace.NOTIFY_EXIT_THREAD")
logger.info("Thread exited - ThreadID: %d", trace.getMeta("ExitThread", -1))
elif event == vtrace.NOTIFY_STEP:
logger.info("vtrace.NOTIFY_STEP")
else:
logger.warning("Unhandled vtrace event type of: %d", event)
class DistributedNotifier(Notifier):
"""
A notifier which will distributed notifications out to
locally registered notifiers so that remote tracer's notifier
callbacks only require once across the wire.
"""
# NOTE: once you turn on vtrace.NOTIFY_ALL it can't be turned back off yet.
def __init__(self):
Notifier.__init__(self)
self.shared = False
self.events = []
self.notifiers = {}
for i in range(vtrace.NOTIFY_MAX):
self.notifiers[i] = []
def notify(self, event, trace):
self.fireNotifiers(event, trace)
def fireNotifiers(self, event, trace):
"""
Fire all our registerd local-notifiers
"""
nlist = self.notifiers.get(vtrace.NOTIFY_ALL, [])
for notifier in nlist:
try:
notifier.handleEvent(event, trace)
except Exception:
logger.error("Exception in notifier:\n%s", traceback.format_exc())
nlist = self.notifiers.get(event, [])
for notifier in nlist:
try:
notifier.handleEvent(event, trace)
except Exception:
logger.error("Exception in notifier:\n%s", traceback.format_exc())
def registerNotifier(self, event, notif):
"""
Register a sub-notifier to get the remote callback's via
our local delivery.
"""
nlist = self.notifiers.get(event)
nlist.append(notif)
def deregisterNotifier(self, event, notif):
nlist = self.notifiers.get(event)
nlist.remove(notif)
class LibraryNotifier(Notifier):
def notify(self, event, trace):
logger.info("LibraryNotifier.notify(%r, %r)", event, trace)
# update unresolved breakpoints:
trace._updateBreakAddresses()
# check meta
if hasattr(trace, 'db'):
cfgBreakLibLoad = trace.db.config.vdb.BreakOnLibraryLoad
cfgBreakLibInit = trace.db.config.vdb.BreakOnLibraryInit
else:
cfgBreakLibLoad = False
cfgBreakLibInit = False
#import envi.interactive as ei; ei.dbg_interact(locals(), globals())
breakLibLoad = trace.getMeta('BreakOnLibraryLoad')
if breakLibLoad or cfgBreakLibLoad:
# stop this instant!
trace.sendBreak()
breakLibInit = trace.getMeta('BreakOnLibraryInit')
if breakLibInit or cfgBreakLibInit:
# add Breakpoint for __entry
libnormname = trace.getMeta('LatestLibraryNorm')
entryname = "%s.__entry" % (libnormname)
logger.debug("BreakOnLibraryInit: %r\t\thooking %s", libnormname, entryname)
# WARNING: this expects all libraries (and binaries) to have a
# __entry. every library *does*, we just need to make sure Viv/
# Vtrace names them appropriately.
try:
initva = trace.parseExpression(entryname)
logger.warning("LoadLibrary(%r): Breakpoint added at 0x%x (%r)", libnormname, initva, entryname)
self._doAddBreakByExp(trace, entryname)
except Exception as e:
logger.warning("LoadLibrary(%r): Can't add breakpoint! %r", libnormname, e)
def _doAddBreakByExp(self, trace, expr):
logger.debug("_doAddBreakByExp(%r, %r)", trace, expr)
trace.addBreakByExpr(expr)