Struggling with accurate Entity Recognition in custom NLP models for advanced semantic content optimization

I'm currently developing a proprietary content optimization algorithm heavily reliant on semantic analysis to enhance our clients' topical authority. We've built a custom NLP pipeline, but I'm encountering significant technical hurdles with the entity recognition phase, particularly when dealing with highly specific, long-tail, or industry-specific entities that aren't well-represented in standard pre-trained models.

• Current Challenge: Achieving satisfactory precision and recall for named entities beyond common knowledge domains. We're seeing a high rate of false positives/negatives and struggle with entity disambiguation in nuanced contexts.
• Specific Problem: Adapting or fine-tuning existing transformer-based models (e.g., BERT, RoBERTa) for niche entity extraction without massive, domain-specific labeled datasets. Also, exploring effective strategies for linking identified entities to a knowledge graph for deeper semantic understanding.
• Core Question: What advanced techniques or frameworks are proving most effective for robust and scalable entity recognition in custom-built content optimization algorithms, especially when dealing with domain-specific lexicon and limited labeled data?

Looking forward to expert insights on this technical block.