tensorflow / neural-structured-learning

Training neural models with structured signals.

1,008 190 Language: Python License: Apache-2.0 Updated: 1mo ago

adversarial-learning graph-learning keras neural-networks regularization structured-signals tensorflow

README

Neural Structured Learning in TensorFlow

Neural Structured Learning (NSL) is a new learning paradigm to train neural
networks by leveraging structured signals in addition to feature inputs.
Structure can be explicit as represented by a graph [1,2,5] or implicit as
induced by adversarial perturbation [3,4].

Structured signals are commonly used to represent relations or similarity among
samples that may be labeled or unlabeled. Leveraging these signals during neural
network training harnesses both labeled and unlabeled data, which can improve
model accuracy, particularly when the amount of labeled data is relatively
small. Additionally, models trained with samples that are generated by
adversarial perturbation have been shown to be robust against malicious
attacks, which are designed to mislead a model's prediction or classification.

NSL generalizes to Neural Graph Learning [1] as well as to Adversarial Learning
[3]. The NSL framework in TensorFlow provides the following easy-to-use APIs and
tools for developers to train models with structured signals:

Keras APIs to enable training with graphs (explicit structure) and
adversarial perturbations (implicit structure).
TF ops and functions to enable training with structure when using
lower-level TensorFlow APIs
Tools to build graphs and construct graph inputs for training

The NSL framework is designed to be flexible and can be used to train any kind
of neural network. For example, feed-forward, convolution, and recurrent neural
networks can all be trained using the NSL framework. In addition to supervised
and semi-supervised learning (a low amount of supervision), NSL can in theory be
generalized to unsupervised learning. Incorporating structured signals is done
only during training, so the performance of the serving/inference workflow
remains unchanged. Please check out our tutorials for a practical introduction
to NSL.

Getting started

You can install the prebuilt NSL pip package by running:

pip install neural-structured-learning

For more detailed instructions on how to install NSL as a package or to build it
from source in various environments, please see the
installation guide

Note that NSL requires a TensorFlow version of 1.15 or higher. NSL also supports
TensorFlow 2.x with the exception of v2.1, which contains a bug that is
incompatible with NSL.

Videos and Colab Tutorials

Get a jump-start on NSL by watching our video series on YouTube! It gives a
complete overview of the framework as well as discusses several aspects of
learning with structured signals.

</a>
<a href="http://www.youtube.com/watch?feature=player_embedded&v=pJRRdtJ-rPU"
target="_blank">

</a>
<a href="http://www.youtube.com/watch?feature=player_embedded&v=3RQqTTOY0U0"
target="_blank">

</a>
<a href="http://www.youtube.com/watch?feature=player_embedded&v=Js2WJkhdU7k"
target="_blank">

</a>

We've also created hands-on colab-based tutorials that will allow you to
interactively explore NSL. Here are a few:

You can find more examples and tutorials under the
examples directory.

Contributing to NSL

Contributions are welcome and highly appreciated - there are several ways to
contribute to TF Neural Structured Learning:

Case studies: If you are interested in applying NSL, consider wrapping up
your usage as a tutorial, a new dataset, or an example model that others
could use for experiments and/or development. The examples
directory could be a good destination for such contributions.
Product excellence: If you are interested in improving NSL's product
excellence and developer experience, the best way is to clone this repo,
make changes directly on the implementation in your local repo, and then
send us pull request to integrate your changes.
New algorithms: If you are interested in developing new algorithms for NSL,
the best way is to study the implementations of NSL libraries, and to think
of extensions to the existing implementation (or alternative approaches). If
you have a proposal for a new algorithm, we recommend starting by staging
your project in the research directory and including a colab
notebook to showcase the new features. If you develop new algorithms in your
own repository, we would be happy to feature pointers to academic
publications and/or repositories using NSL from this repository.

Please be sure to review the contribution guidelines.

Research

See our research directory for research projects in Neural
Structured Learning:

Featured Usage

Please see the usage page to learn more about how NSL is being
discussed and used in the open source community.

Issues, Questions, and Feedback

Please use
GitHub issues
to file issues, bugs, and feature requests. For questions, please direct them to
Stack Overflow with the
"nsl" tag. For feedback,
please fill this
form;
we would love to hear from you.