<p>I have no knowledge of embedding and neural networks, but formatting the question is already helping a lot to clarify a question.</p>
<p>Here is an example with the following changes</p>
<ul>
<li>place code at the end</li>
<li>use capitals</li>
<li>add section headings</li>
<li>turn the long paragraph into points</li>
<li>remove stuff about the error message</li>
<li>remove repetitive use of 'please' and 'tell me'</li>
<li>highlighted question</li>
<li>removed question about positional embedding to improve focus</li>
</ul>
<p>As I understood from comments by others who know this stuff better there might also be some XY problem underlying the closure and embedding float numbers is something you do not do. To clarify why you are thinking about doing this at all is to explain why you have this problem. For instance <em>&quot;question: can I add up 1+1 = 3? answer: no, but why would you want to do that? reply: because my keyboards '2' is stuck.&quot;</em> To avoid that start with what you are doing.</p>
<hr />
<p>My question relates to pytorch code of transformers suitable for machine translation. (See the code at the end of the question)</p>
<p>One the first steps is embedding the word vectors. But for none word integer purposes, is it possible to also use <code>nn.Embedding</code> or something alike so to have float numbers?</p>
<p><strong>Why does <code>nn.Embedding</code> only deal with words? What are alternative ways to embed floating numbers which are commonly used (e.g in non-machine-translation, non-integer tasks like most of neural nets, time series analysis, CNNs etc)?</strong></p>
<p>I know float numbers don't work with nn.embedding. My question is when we are dealing with integers, someway we embed them but what is usually done equivalently when we are working with float numbers?</p>
<ul>
<li>One of possible ways is to use fully connected layer but in machine translation they also can use indexes(in form of float) and map them to another float vector ,except and avoid using word embeddings or in case of general (nn.embedding).</li>
<li>They probably trying to preserve some features of language and so thats why they use word2vec embeddings rather than using vanilla fully connected layers.</li>
<li>But, in terms of using transformer models to get float inputs, we don't know what should be preserved</li>
<li>So my question is what the other researchers of this field have done?</li>
</ul>
<h3>Example code</h3>
<pre><code>import torch
import torch.nn as nn
class Encoder(nn.Module):
    def __init__(
        self,
        src_vocab_size,
        device,
        embed_size):
        super(Encoder, self).__init__()
        self.embed_size = embed_size
        self.device = device
        self.word_embedding = nn.Embedding(src_vocab_size, embed_size)
    def forward(self, x):
        out = self.word_embedding(x)
        print(out)
        
if __name__ == &quot;__main__&quot;:
    device = torch.device(&quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;)
    x = torch.tensor([[1, 5, 6, 4, 3, 9, 5, 2, 0], [1, 8, 7, 3, 4, 5, 6, 7, 2]]).to(device )
    src_vocab_size = 10
    embed_size=512
    model = Encoder(src_vocab_size,device,embed_size).to(device)
    out = model(x)
<span class="math-container">```</span>
</code></pre>
