SJTU SPARK

    Weibo analytics

    Interesting Weibo statistics after operations on Spark Framework

    Here are some interesting characteristics discovered from the online Weibo community. Our datasets include more than 1 million users along with 57 million Weibos communications.

    Details

    Acemap

    Large-scale Academic Recommender System

    In this project, we build an academic research engine which can: return paper search results based on topic similarity with user's query; analyse the latent topic distribution and topic development over time; visualize the "topic tree" starting from a particular paper.

    Details

    GraphX

    Interactive Big Graph Algorithms on Spark

    Graph algorithms implemented in GraphX on Spark Cluster. Firstly, the project provides an overall review on the graphx-related algorithms and tutorials on basics of GraphX. Furthermore, we implement algorithms in C/C++ and compare the performance with GraphX. All writen programs are documented with stantdard annotations and examples.

    Details