multihead-attention

Semantic segmentation is an important job in computer vision, and its applications have grown in popularity over the last decade.We grouped the publications that used various forms of segmentation in this repository. Particularly, every paper is built on a transformer.

computer-vision transformer segmentation semantic-segmentation encoder-decoder instance-segmentation multihead-attention self-attention

Updated Sep 6, 2024

MirunaPislar / multi-head-attention-labeller

Star

Joint text classification on multiple levels with multiple labels, using a multi-head attention mechanism to wire two prediction tasks together.

sentiment-analysis transformer semi-supervised-learning sentence-classification attention-mechanism joint-models zero-shot-learning multi-task-learning conll-2003 bilstm error-detection multihead-attention joint-learning sequence-labelling bilstm-attention hedge-detection

Updated Oct 17, 2020
Python

iafarhan / causal-synthesizer-multihead-attention

Star

Synthesizer Self-Attention is a very recent alternative to causal self-attention that has potential benefits by removing this dot product.

python pytorch attention multihead-attention synthesizer-attention

Updated Apr 21, 2022
Python

hrithickcodes / transformer-tf

Star

This repository contains the code for the paper "Attention Is All You Need" i.e The Transformer.

transformers neural-machine-translation attention-is-all-you-need multihead-attention self-attention transformer-architecture transformer-tensorflow2

Updated Dec 6, 2022
Jupyter Notebook

shawnhan108 / AutoTruckX

Star

An experimental project for autonomous vehicle driving perception with steering angle prediction and semantic segmentation using a combination of UNet, attention and transformers.

transformer attention transfer-learning autonomous-driving autonomous-vehicles unet semantic-segmentation udacity-self-driving-car resnet-50 multihead-attention cnn-lstm unet-image-segmentation conv3d steering-angle-prediction setr

Updated Mar 27, 2021
Python

bkhanal-11 / transformers

Star

The implementation of transformer as presented in the paper "Attention is all you need" from scratch.

transformers attention-mechanism attention-is-all-you-need multihead-attention self-attention positional-encoding

Updated Mar 4, 2023
Python

jaydeepthik / Nano-GPT

Star

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

transformers pytorch gpt multihead-attention pytorch-implementation

Updated May 10, 2023
Jupyter Notebook

antonio-f / GPT_from_scratch

Star

Very simple implementation of GPT architecture using PyTorch and Jupyter.

python tutorial simple jupyter-notebook pytorch transformer easy-to-use gpt newbie from-scratch noob-friendly bigram-model multihead-attention self-attention triangular-matrix

Updated Apr 17, 2023
Jupyter Notebook

abhilash1910 / GraphAttentionNetworks

Sponsor

Star

This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.

tf2 keras-tensorflow leaky-relu graph-attention-networks multihead-attention self-attention