注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

gmd20的个人空间

// 编程和生活

 
 
 

日志

 
 

Apache Kafka 看上去一个很不错的分布式消息通讯框架  

2013-01-25 01:49:11|  分类: 程序设计 |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |
最初是LInkedin用于log 消息的记录的,看看能不能从中学到一些怎么写log文件等日志系统方面的技术。

A high-throughput distributed messaging system
http://kafka.apache.org/performance.html

设计思想
http://kafka.apache.org/design.html

Major Design Elements

There is a small number of major design decisions that make Kafka different from most other messaging systems:

1.Kafka is designed for persistent messages as the common case
2.Throughput rather than features are the primary design constraint
3.State about what has been consumed is maintained as part of the consumer not the server
4. Kafka is explicitly distributed. It is assumed that producers, brokers, and consumers are all spread over multiple machines.


相关 论文和介绍文档
Kafka papers and presentations
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations

比如

Kafka: a Distributed Messaging System for Log Processing

http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf

Building LinkedIn’s Real-time Activity Data Pipeline

http://sites.computer.org/debull/A12june/pipeline.pdf



其他log收集系统
Facebook’s Scribe [6], Yahoo’s Data 
Highway [4], and Cloudera’s Flume [3].

Those systems are 
primarily designed for collecting and loading the log data into a 
data warehouse or Hadoop [8] for offline consumption.



一年前就读过kafka的文章啊,忘的差不多了
http://hi.baidu.com/widebright/item/9265c70c99e94ef3a01034c1
  评论这张
 
阅读(581)| 评论(0)
推荐 转载

历史上的今天

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017