SJTU-Spark
Rome was not built in one day, but Spark can be.

Team

Advisors


  • Prof. Li SONG

  • Prof. Xinbing WANG

  • Prof. Guihai CHEN



  • Prof. Xiaohua TIAN

  • Prof. Weijie WU



    • Students

    Chunnan YAO
    Hangchen YU
    Zhouyihai DING
    Ge CHEN
    You WU
    Xingbang LIU
    Rongqi YAN
    Yuchuan GOU
    Wenguang HUANG
    Yang LI
    Chang ZHOU
    Jialu ZHANG

    Cluster

    Hardware Layer

  • 5 nodes
  • 25 cores
  • 125GB Memory
  • 500GB Disk/node

    • Software Layer

  • Spark Standalone
  • Hadoop
  • HDFS,Hbase
  • Yarn/Mesos

    • Datasets

  • 10MB MovieLens
  • 2GB SougouDict
  • 15GB Sina Weibo
  • 50GB Sina Topics



    • Motivation

    Fast & Light Distributed Computing

    Real-time Large-Scale data mining

    Fast Huge-Sparse-Matrix operations

    Quick Machine Learning Algorithm

    Simple Usage & Quick Deployment

    Contact us