Challenges of Open Data and Collaborative Resources

Language Technology (LT) is a data-intensive field and major breakthroughs have stemmed from a better use of Language Resources (LR). The LR field is very active, but it needs coherence. FLaReNet – Fostering Language Resources Network (http://www.flarenet.eu/) produced “The Strategic Language Resource Agenda”, a plan for actions and infrastructures for future initiatives in the field. Recognising that the development of LTs is conditioned by various factors, the FLaReNet recommendations are organised around nine dimensions: a) Infrastructure, b) Documentation, c) Development, d) Interoperability, e) Coverage, Quality and Adequacy, f) Availability, Sharing and Distribution, g) Sustainability, h) Recognition and i) International cooperation. Taken together these directions contribute to a sustainable LR ecosystem. They are relevant also with respect to the challenges of Big Open Data and collaborative resource construction. An implication of collaboration is that interoperability acquires even more value. The same is true for sustainability, for data infrastructure enabling international collaboration, and also for notions such as authority and trust.

The traditional LR production process is too costly. A new paradigm is pushing towards open, distributed language infrastructures based on sharing LRs, services and tools. Joining forces in big experiments that collect thousands of researchers is the only way for our field to achieve the status of a mature science. This will serve better the needs of language applications, enabling building on each other achievements, integrating results, and having them accessible to various systems, thus coping with the need of more ‘knowledge intensive’ LRs for effective multilingual content processing. I will briefly mention the LRE Map that collect collaboratively built information on LRs, with the involvement of all the community. An important next step (already in process) is to connect these LR metadata and processed language data to Linguistic Linked Open Data (LLOD).

Technical scientific issues are obviously important, but organisational, coordination, political issues play a major role in our field as in every other. One of the challenges for the collaborative model to succeed will be to ensure that the community is engaged at large! This can also be seen as an effort to push towards a culture of "service to the community" where everyone has to contribute. This “cultural change” is not a minor issue.