SpeechLock: voice to vector, voice similarity

SpeechLock

Идея

Не еш меня, подумой

same person ✓

not same person ✘

Hello, I’m Tvorozhek.Let me in, please.

тыгыдык-тыгыдык гыы я лошадка

Что тут вообще происходит?

Датасет

http://forvo.comСкачано 7406 произношений.

Самые популярные слова:

1 66привет 52здравствуйте 27машина 26русский 21Китай 20медведь 19счастье 19котёнок 19

Первая попытка

Предобработка голосов

1. Убрали все звуки длиною меньше секунды

2. Выбрали случайную секунду

3. Нарезали на кусочки по 25 мс с пересечениями по 15 мс

4. Применили преобразование Фурье к каждому кусочку

Преобразование Фурье позволяет получить “фичи” голосов

до Фурье

после Фурье

Структура нейросети

Речь голоса 1

Речь голоса X

Conv2D MaxPool2D Dense

Concatsame weights not same w.

похоже? не похоже?

batch size x sound channels x pieces x fft sound

Вторая попытка

Спектрограммы

По горизонтали время.

По вертикали частоты.

Цвет означает амплитуды, соответствующие определенной частоте. yulia_m: кусь

yulia_m: ветчина

Структура нейросети

Conv1D MaxPool1D GlobalPool Dense

Речь с голосом 1

Другая речь с голосом 1

Речь с голосом X

Речь с голосом 1, как вектор

Речь с голосом X, как вектор

n filters = 100filter size = 5leaky rectify

pool size = 2 n units = 300

batch size x frequency x time

размерность векторов = 150

Другая речь с голосом 1, как

вектор

Функция потерь на следующем слайде >>>

Функция потерь

Функция предсказания

64%AUC ROC

Похожие работы

FaceNet: A Unified Embedding for Face Recognition and Clustering https://arxiv.org/pdf/1503.03832v3.pdf

Recommending music on Spotify with deep learning http://benanne.github.io/2014/08/05/spotify-cnns.html

Ссылки

github.com/xenx/speech

Презентация на гитхабе pdf

Поставь звездочку!

SpeechLock: voice to vector, voice similarity

Software

IR Models: The Vector Space Modelian/irF02/lectures/07Models...IR Models: The Vector Space Model Lecture 7 Lecture 7 Information Retrieval 2 Boolean Model Disadvantages Similarity

NRU ITMO, St Petersburg, 2018 Vector semantics -III ... · word vectors representations training algorithm, demonstrating that vectors 1) allow to estimate words similarity reasonably

Database Management Systems, R. Ramakrishnan1 Computing Relevance, Similarity: The Vector Space Model Chapter 27, Part B Based on Larson and Hearst’s slides

Vector Semantics Introduction. Dan Jurafsky Why vector models of meaning? computing the similarity between words “fast” is similar to “rapid” “tall” is

Ontology Optimization Algorithm for Similarity Measuring ...ontology vector is obtained in terms of Lagrangian relaxation approach. Finally, four experiments are presented from various

Cosine similarity metric calculation on low power ... fileterm 1 sentence 1 Vector Space Model Term Frequency— Inverse Document Frequency (tf — = Cosine similarity cos 9 = x log

Semantic similarity, vector space models and word- sense disambiguation Corpora and Statistical Methods Lecture 6

1 Computing Relevance, Similarity: The Vector Space Model

Vector Semantics - Carnegie Mellon Universitydemo.clab.cs.cmu.edu/NLP/F19/files/slides/17-embeddings.pdf2 Word similarity for plagiarism detecton Word similarity for historical linguistcs:

Creating a Feature Vector to Identify Similarity between ... · Creating a Feature Vector to Identify Similarity between MIDI Files ... that share musical characteristics, ... and

IntelliGO: a new vector-based semantic similarity measure ... · only a few of the semantic similarity measures described so far can handle GO annotations di erently according to

Ch. 11 - Similarity Class Notes. What Is Similarity?

A Similarity Learning Approach to Content Based Image ... · for image similarity, we propose an approach in which machine-learning algorithms (neural networks and support vector

Symbolic Melodic Similarity (through Shape Similarity)

Information Retrieval Basic Document Scoring. Similarity between binary vectors Document is binary vector X,Y in {0,1} v Score: overlap measure What’s

Using Randomization to Attack Similarity Digests · Using Randomization to Attack Similarity Digests ... Using Randomization to Attack Similarity ... Similarity digest schemes exhibit

Algebra II, Module 2€¦ · Lesson 13: Properties of Similarity Transformations Exit Ticket A similarity transformation consists of a translation along the vector 𝐹𝐹 𝐺𝐺

Similarity and Transformations...Holt McDougal Geometry Similarity and Transformations A transformation that produces similar figures is a similarity transformation. A similarity transformation

IntelliGO: a new vector-based semantic similarity measure ... · IntelliGO: a new vector-based semantic similarity measure including annotation origin Sidahmed Benabderrahmane1*,

Chapter 6: Similarity Perform Similarity Transformations 6.7