I believe I finally have a final architecture for Webseer. Inspired by the architecture of Nutch, I have ripped almost everything out of Webseer and turned it into plugins. It turns out I couldn't use all the IO stuff in Nutch, since their APIs are set up to handle text parsing only, so I have defined the following extension points that define the core of Webseer:
FeatureExtractor - a visitor that collects features on a model
DataSourceIO - turns bytes into models
Transformer - transforms models into other models
I haven't figured out the classification side yet, but I need to abstract the common classification tasks into an interface. Something with more a little more power than just being able to plug in weka classes.
Sunday, October 7, 2007
Subscribe to:
Posts (Atom)