I am building an ai powered language learning app with features like real time pronunciation, more listening and speaking practice, and personalizes session planning. Anyone interested?? Or any suggestions??

I am trying to build an nmt for prakrit languages. But I am having trouble finding the datasets. What must be the minimum threshold for the data size to get a descent BLEU score let's say around 30. You can also refer my earlier project I have posted in this subreddit.

Hi everyone,

I'm working on a project where I need to extract information from bills, questionnaires, and other documents to complete a structured report on an organization's climate transition plan. The report includes placeholders that need to be filled based on the extracted information.

For context, the report follows a structured template, including statements like:

I need to rewrite all of those statements and merge them in the form a final, complete report. The challenge is that the placeholders must be filled based on answers to a set of decision-tree-style questions. For example:

1.1 Does the organization have a climate transition plan? (Yes/No)

• If Yes → Go to question 1.2
• If No → Skip to question 2

1.2 Is the transition plan approved by administrative bodies? (Yes/No)

• Regardless, proceed to 1.3

1.3 Are the emission reduction targets aligned with limiting global warming to 1.5°C? (Yes/No)

• Regardless, reference supporting evidence

And so on, leading to more questions and open-ended responses like:

• "Explain how locked-in emissions impact the organization's ability to meet its emission reduction targets."
• "Describe the organization's strategies to manage locked-in emissions."

The extracted information from the bills and questionnaires will be used to answer these questions. However, my main issue is designing a method to take this extracted information and systematically map it to the placeholders in the report based on the decision tree.

I have an idea in mind, but always like to have others' insights. Would appreciate your opinion on:

1. Structuring the logic to take extracted data and answer the decision-tree questions reliably.
2. Mapping answers to the corresponding sections of the report.
3. Automating the process where possible (e.g., using rules, NLP, or other techniques).

Has anyone worked on something similar? What approaches would you recommend for efficiently structuring and automating this process?

Thanks in advance!

I'm looking for an AI solution to an issue I face pretty regularly. I run surveys and receive many open-end text responses. Sometimes there are up to 3k of these responses. From these responses, I need to find overarching themes that encompass the sentiment of the open-end text responses. Doing it manually in a team is an absolute pain as it involves reading each response individually and categorizing it in a theme manually. This takes a lot of time.

I've tried using ChatGPT 4-o and other specialized GPTs within the ChatGPT interface to try this but they do not work well. It randomly categorizes options after a point and only does the first 30-40 responses well. It also fails to recognize responses that have typos. Any solutions or specific tools you would recommend? My friend and I know how to code as well and would be open to using APIs, but ready to go services would be better.

So the project is we get bunch of unstructured data like emails etc and we have to extract data from it like name, age and in case of order mails things like quantity, company name etc. I think Named Entity Recognition is the way to go but am stuck on how to proceed. Any help would be appreciated. Thank you

Edit: I know that we have can use NER but how do I extract things like quantity, item name etc apart from tags like Person, Location etc. Thanks

Hello,

I want to do NER on texts using this model: https://huggingface.co/urchade/gliner_large_bio-v0.1 . The texts I am working with are of variable length. I do not truncate or split them. The model seems to have run fine on them, except it displayed warnings like:

UserWarning: The sentencepiece tokenizer that you are converting to a fast tokenizer uses the b
yte fallback option which is not implemented in the fast tokenizers. In practice this means that the fast version of the tokenizer can produce unknown tokens whereas the sentencepiece version would have converted these
unknown tokens into a sequence of byte tokens matching the original piece of text.
 warnings.warn(
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.

I manually gave a max_length longer than what was in the config file:

model_name = "urchade/gliner_large_bio-v0.1"model = GLiNER.from_pretrained(pretrained_model_name_or_path=model_name, max_length=2048)

What could be the consequences of this?

Thank you!

so i dont really know if this is the right place to ask so if this is not the right place to ask this please point me to where is the most appropriate. with that said

my goal is to create my own japanese to english translator tool. i know japanese so even if the tool that i create isnt optimal it would be easy for me to correct.

what tools do i need to do to achieve my goal? does that tool also have a way to visualize the flow of the conversion through maybe a flowvhart? if not im fine with not having that feature.

also might be offtopic but is there a info on net where it shows you how the translator(machine or program) breaks down the sentence and translate it? interested in japanese text

I have an idea to replace the Transformer Structure, here is a short explaination.

In Transformer architicture, it uses weights to select values to generate new value, but if we do it this way, the new value is not percise enough. 

Assume the input vectors has length N. In this method, It first uses a special RNN unit to go over all the inputs of the sequence, and generates an embedding with length M. Then, it does a linear transformation using this embedding with a matirx of shape (N X N) X  M.

Next, reshape the resulting vector to a matrix with shape N x N. This matrix is dynamic, its values depends on the inputs, whereas the previous (N X N) X  M matrix is fixed and trained.

Then, times all input vectors with the matrix to output new vectors with length N.

All the steps above is one layer of the structure, and can be repeated many times.

After several layers, concatanate the output of all the layers. if you have Z layers, the length of the new vector will be ZN.

Finally, use the special RNN unit to process the whole sequence to give the final result(after adding several Dense layers).

The full detail is in this code, including how the RNN unit works and how positional encoding is added: 

https://github.com/yanlong5/loong_style_model/blob/main/loong_style_model.ipynb

Contact me if you are interested in the algorithm, My name is Yanlong and my email is y35lyu@uwaterloo.ca

Hi, all.

This might be a long shot. I have some old annotation in .ann. My brat installation used to work. But I have tried multiple ways to install brat on both mac and linux server from source code and image, but all failed. It seems to be some cgi issue.

Since I haven't seen the source code updated for many years, I am not sure if it is still installable. If it can be installed, which source code/docker image has been proven to be working?

thanks!

Hi everyone.

I will submit for the first time to the ARR feb cycle including ACL conference.

The ACL 2025 website regulation states that long paper is up to 8 pages, so can't it be over 1-2 pages?

In fact, long papers in ACL, EMNLP, and NAACL conf have often been 9 to 10 pages.

Hello dear community!
I’m working with a dataset of job postings for data scientists. One of the columns contains the "required skills." I’d like to analyze this dataset using topic modeling to extract the most prominent skills and skill clusters.

The data looks like this:
"3+ years of experience with data exploration, data cleaning, data analysis, data visualization, or data mining. 3+ years of experience with statistical and general-purpose programming languages for data analysis. [...]"

I tried using BERTopic with "normal" embeddings and more tech focused embeddings but got very bad results. I am not experienced with Topic Modeling. I am glad for any help :)

Hi everyone, I’m an international student who recently graduated from the University of Edinburgh with a Master’s degree (Merit) in a field related to NLP and Machine Learning. My undergraduate background is in linguistics. After graduation, I noticed that finding a MLE role in the UK often requires a PhD. However, after discussing with my supervisor, she suggested that I consider applying for a RA position first, as the PhD application process is highly competitive.

I’m unsure about the best path forward and would appreciate some advice. Should I focus on finding an NLP-related RA position in the UK and then apply for a PhD? Or would it make more sense to first transition into a SDE role, gain industry experience, and later pivot to MLE before applying for a PhD based on my work experience? Alternatively, should I reconsider pursuing a PhD altogether?

Feel free to ask me for more information if it's needed for suggestions! Also appreciate if there is any lab or uni recommendations for RA/Phd.

FYI, I don't have any work experiences so far, only research experiences in linguistics and NLP.

I am just wondering what avenues of research or what topic to do research on if we have advanced NLP models like Chat GPT and Gemini who have enormous processing power and training data access, I mean isn't the research useless if whatever we do Chat GPT can do better?

Hi!

I'm a 17 years old guy from Moscow, in the 10th grade, and I'm planning to apply to either HSE (Higher School of Economics) or Moscow State University (MSU) for a program in Fundamental and Applied/Computational Linguistics. To do this, I'm planning to take the Unified State Exam (USE) in advanced mathematics, computer science, and English, as well as study some topics from the first-year curriculum in advance. I'm already gradually practicing programming in Python, advanced math (I'm currently reading about limits and integrals), and slowly getting into the basics of linguistics. I also want to start learning a second foreign language, which is mandatory in both universities. However, I don't know which one would be better. Both universities offer a choice of European and Asian languages.

It's important to me that the third language would be a good addition to my future resume or be in demand in NLP.

I'm not afraid of any difficulties. I'm ready for any challenges if I approach them at my own pace, I'm ready to adapt my mindset. I'm left-handed, so writing from right to left is not difficult for me, I tried it. Logograms are not a catastrophe for me to memorize as well. In fact, I love making up my own writing systems just for fun.

Which language would you choose and why?

Thank you!

First time researcher here. I got assessment scores of 1.5, 1.5 and 2 from three reviewers. All the reviewers acknowledge the novelty of my work in strenghts. But the points reviewers raised in weakness if addressed will increase the paper length from short to long (as this was mainly an initial study as mentioned in limitations). Also reviewers dont seem to understand the point of paper.For such a low score, is their any point for doubling down on convincing reviewers or should I just acknowledge their criticism and improve in another submission? Also what should be my target scores for acceptance into a relevant ACL workshop?

Anyone know what average overall score required to be accepted to main, or like what is a safe number? Is there anywhere I can see average scores for the October cycle?

Hi!

I've been invited to an interview for the MSc in Speech and Language Processing at Ediburgh. I've never done an interview for a program before so I'm unsure about what they would ask or about the organization of the interview.

Has anyone done an interview for this program or other related?

Any advice on the interview topic is welcomed!

I mean for mainly business purposes, e.g., decks, content, reports, etc. Can AI do it well? Will it make bad mistakes? Should I use a person instead?

I’ve been sifting through the website but cannot find some pretty basic info about the program details, such as application deadlines and if GREs are required. Has anyone studied or at least applied to UP Cité? I would really appreciate any help or direction. I’m coming from an unrelated area of study, if that helps at all. Thank you in advance.

Hi guys,

* Will converting SQL tables into embeddings, and then retreiving query from them will be of help here?

* How do I make sure my chatbot understands the context and asks follow-up questions if there is any missing information in the user prompt?

* How do I save all the user prompt and response in one chat so as to make context of the chat history? Will not the token limit of the prompt exceed? How to combat this?

* What are some of the existing open source (langchains') agents/classes that can be actually helpful?

**I have tried create_sql_query_chain - not much of help in understanding context

**create_sql_agent gives error when data in some column is of some other format and is not utf-8 encoded [Also not sure how does this class internally works]

* Guys, please suggest me any handy repository that has implemented similar stuff, or maybe some youtube video or anything works!! Any suggestions would be appreciated!!

Pls free to dm if you have worked on similar project!

hey there!

I am currently looking for an MA program in Computer linguistics/ Language and AI or other programs that would connect IT with linguistics, yet I don’t have any previous experience in programming. Anyone knows about the programs in Europe (and the UK) which would accept applicants with various backgrounds without prior knowledge in IT? That would immensely help me.

Please, let me know if you’re by any chance aware of scholarships available for these countries/programs ✨✨

Thank you a lot in advance!

Hello everyone. I am newbie in NLP world, and have a task from one firm. It is technical task for intern position. Here is the description of the task:

You task it to process provided technical articles and implement continual training for one of the large Language Models – BERT. The purpose is such that your BERT model understands the context of those papers and ready to answer questions related to those papers. For that, you need to work with Hugging Face. It is also suggested for you to work via Colab. Your deliverables are:

·       Deploy original BERT model and test it by asking the questions

·       Do continual training of BERT and generate a code allowing to ask questions regarding paper context

·       Compare answers of original and your BERT models and show that your model is fit-to-purpose

Here is my problem. As I know, when we finetune BERT we need question, answer, context, start and end positions of answer. But there are too many content provided by them. 6 pdfs which are separated books. Is there a way to generate that questions answers and etc in easy way?

I am looking for currently running or future shared tasks in NLP .

I've had success in the past with BERT and with the release of ModernBert I have substituted the new version. However, the results are nowhere near as good. Previously, finetuning a domain adapted BERT model would achieve an f1 score of ~.65, however swapping out for ModernBERT, the best I can achieve is an f1 score of ~.54.

For context, as part of my role as an analyst I partially automate thematic analysis of short text (between sentence and paragraphs). The data is pretty imbalanced and there are roughly 30 different labels with some ambiguous boundaries.

I am curious if anyone is experiencing the same? Could it be the the long-short attention isn't as useful for only shorter texts?

I haven't run an exhaustive hyperparameter search, but was hoping to gauge others' experience before embarking down the rabbit hole.

Hello,

New to NLP and looking for a multilingual dataset/corpus (That won't crash my computer) that allows for a model to be trained that will predict the next character in a sequence. Thanks!