GSoC '26 Week 3: Implementing the Blended Math and the need for a Validation Layer

Last week I was able to figure out the current flaws of step 2 of the pipeline. Sometimes, even though the math is correct, we get the wrong results. This issue is termed “Context Drowning.”

To solve this, I’ve come up with a solution: Semantic Self Learning and Entity Disambiguation.

Right now, what we are doing in step 2 is, for both the resources in the raw triple, we fetch the top 15 URIs using the DBpedia lookup API. And then we calculate cosine similarity on the abstracts using MiniLM and rank them on the basis of this cosine similarity score.

Now in the optimized solution, we are calculating this score using an 80/20 scoring blend. The final URI score is a blend of 80% Abstract Vector Similarity and 20% Lexical Edit Distance.

We can’t 100% rely on the Abstract Vector Similarity; this was proved by the Mia Wallace example which I mentioned in last week’s blog. If the wrong entity contains too many words that match the context of the sentence, the model tends to give it a high score, even though it’s the wrong entity.

So we also look at the name of the entity. There should be some similarity between the actual resource (in the raw triple generated) and the names in the extracted URI. If that doesn’t match, the score is reduced.

Now because of this small tweak, we are able to correctly eliminate the imposters, who might get a higher score than the actual correct entity. I checked this approach on a few sentences and we are able to map the subject and object in the raw triple to the correct DBpedia URI.

Here’s the output snippet:

Now as we are done with step 2 of the pipeline, the third step is Predicate Linking. After linking the subject and object to the correct URI, we need to find the relation between them—the ontology.

For this, we are using the DBpedia .owl file. It contains around 1,200 properties. We parse that file and find the predicate that correctly fits in the system.

I faced the problem of hallucinations here as well. Sometimes the model hallucinates and picks the wrong ontology. In one of the cases, it mapped “starred in” to something called dbo:BaftaAward, which had nothing to do with the sentence, and the overall final triple was looking meaningless.

To fix this, I again implemented the technique of a Semantic-Lexical Blend, but the ratio this time was 50/50.

The reason for that is that the predicates often resemble a lot with the raw names extracted. For example: (“was written by” -> dbo:writer), (“starred in” -> dbo:starring), and (“director of” -> dbo:director).

Due to this resonance with the raw words, we need to give the names of the predicate a bigger weightage than 20%. So now, our model selects the predicate not only on the basis of the vector similarity (what is in the properties of the predicates), but also what their names are.

This tweak really solved the issue of hallucinations in step 3, predicate linking. Now we are able to rightly identify the relation between two URIs.

With step 2 and 3 now done, take a look at the final triples generated by the model:

But I noticed one thing. Even after applying these dynamic approaches in step 2 and 3, there were still some errors in the final triple generated. We can’t just completely rely on the final output; there has to be some sort of validation layer in the end.

See in this output snippet below, in this sentence the object is George Lucas. But even after all the optimized layers, the final output triple by the model is giving George W. Lucas, which is a historical figure and has nothing to do with the sentence. This is another hallucination of the system; it was not able to figure out the correct George Lucas and mapped to the wrong URI.

And that is why a validation layer is needed for a final check before throwing the final triple as an output.

The role of the validation layer will be to check the correctness of the final triple generated, and if there is a mistake, then report that mistake and trigger a self-learning loop to correct it.

So now I had to brainstorm and figure out what needs to be done in this layer and how to implement this.