Codes

  Home     MCADS Lab     People     Publications     Activities     Codes     Data     Teaching

Weakly-Supervised Action Segmentation and Alignment via Transcript-Aware Union-of-Subspaces Learning

Interaction Compass: Multi-Label Zero-Shot Learning of Human-Object Interactions

Learning to Segment Actions from Visual and Language Instructions

  • We address the problem of unsupervised action segmentation and feature learning in procedural task videos using both visual and language instructions. Our key contributions include proposing i) a Soft Ordered Prototype Learning (SOPL) method for learning visual and linguistic prototype sequences corresponding to subtasks (key-steps) in videos, and ii) a new class of alignment methods, referred to as Differentiable Weak Sequence Alignment (DWSA), for weak alignment of visual and linguistic prototype sequences while allowing self-supervised feature learning.

  • We provide the Python/Pytorch implementation of the method. When using the code in your research work, you should cite the following papers:

    Y. Shen, L. Wang, E. Elhamifar, Learning to Segment Actions from Visual and Language Instructions via Differentiable Weak Sequence Alignment,
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.



  • Project Page

Introvert: Human Trajectory Prediction via conditional 3D attention

  • We developed Introvert, a deep model that predicts human trajectory and path based on their observed trajectory and the dynamic scene context, captured via a conditional 3D visual attention mechanism.

  • We provide the Python/Pytorch implementation of the method. When using the code in your research work, you should cite the following papers:

    N. Shafiee, T. Padir and E. Elhamifar, Introvert: Human Trajectory Prediction via Conditional 3D Attention,
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.



  • Project Page

Compositional Zero-Shot Learning via Fine-Grained Dense Feature Composition

Unsupervised Procedure Learning from Instructional Videos

Shared Attention for Multi-label Zero-shot Learning

  • We address zero-shot multi-label learning for recognition of a large number of seen and unseen labels using a shared multi-attention method with a novel training mechanism.

  • We provide the Python/Tensorflow implementation of LESA. When using the code in your research work, you should cite the following paper:

    D. Huynh, E. Elhamifar, A Shared Multi-Attention Framework for Multi-Label Zero-Shot Learning,
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

  • Project Page

Fine-Grained Generalized Zero-Shot Learning via Dense Attribute-Based Attention

  • We develop a zero-shot fine-grained recognition method with the ability to localize attributes using a dense attribute-based attention and embedding mechanism.

  • We provide the Python/Pytorch implementation of DAZLE. When using the code in your research work, you should cite the following paper:

    D. Huynh, E. Elhamifar, Fine-Grained Generalized Zero-Shot Learning via Dense Attribute-Based Attention,
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

  • Project Page

Interactive Multi-Label CNN Learning with Partial Labels

  • We address efficient end-to-end learning of multi-label CNN classifiers with partial labels using an interactive learning framework.

  • We provide the Python/Tensorflow implementation of IMCL. When using the code in your research work, you should cite the following paper:

    D. Huynh, E. Elhamifar, Interactive Multi-Label CNN Learning with Partial Labels,
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

  • Project Page

Deep Summarization via Supervised Facility Location (SupFL)

Dissimilarity-based Sparse Subset Selection (DS3)

  • Dissimilarity-based Sparse Subset Selection (DS3) is an algorithm based on simultaneous sparse recovery for finding data/model representatives from a large collection of data/models.

  • We provide a MATLAB implementation of DS3. When using the code in your research work, you should cite the following paper:

    E. Elhamifar, G. Sapiro and S. Sastry, Dissimilarity-based Sparse Subset Selection,
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016.

  • Download Code

Sparse Subspace Clustering (SSC)

  • Sparse Subspace Clustering (SSC) is an algorithm based on sparse representation theory for segmentation of data lying in a union of subspaces. For more information please visit the SSC research page.

  • We provide a MATLAB implementation of SSC. When using the code in your research work, you should cite the following paper:

    E. Elhamifar and R. Vidal, Sparse Subspace Clustering: Algorithm, Theory, and Applications,
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2013.

  • Download Code (ADMM)
    Download Code (CVX)

Sparse Manifold Clustering and Embedding (SMCE)

  • Sparse Manifold Clustering and Embedding (SMCE) is an algorithm based on sparse representation theory for clustering and dimensionality reduction of data lying in a union of nonlinear manifolds.

  • We provide a MATLAB implementation of SMCE algorithm. When using the code in your research work, you should cite the following paper:

    E. Elhamifar and R. Vidal, Sparse Manifold Clustering and Embedding,
    Neural Information Processing Systems (NIPS), 2011.

  • Download Code

Sparse Modeling Representative Selection (SMRS)

  • Sparse Modeling Representative Selection (SMRS) is an algorithm based on sparse multiple-measurement-vector recovery theory for selecting a subset of data points as the representatives.

  • We provide a MATLAB implementation of SMRS algorithm. When using the code in your research work, you should cite the following paper:

    E. Elhamifar, G. Sapiro, and R. Vidal, See All by Looking at A Few: Sparse Modeling for Finding Representative Objects,
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012.

  • Download Code

Block-Sparse Subspace Classification (BSSC)

  • Structured-Sparse Subspace Classification is an algorithm based on block-sparse representation techniques for classifying multi-subspace data, where the training data in each class lie in a union of subspaces.

  • We provide a MATLAB implementation of the algorithm. When using the code in your research work, you should cite the following paper:

    E. Elhamifar and R. Vidal, Robust Classification using Structured Sparse Representation,
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011.

  • Download Code