Configuration

Stream produced in two D.C.‘s Need merged stream in DC1 and mirror in DC2

i want to run databus in two clusters dc1, dc2 in different data centers. Stream “example_stream” is produced in dc1 and dc2. Merged stream view across both clusters is required in DC1 and DC2 will be mirror of DC1 for example_stream.Also need local stream views in each Cluster The following configuration explains this scenario

<databus>
<defaults>
  <rootdir>/databus</rootdir>
</defaults>
<streams>
  <stream name="example_stream">
    <sources>
      <source>
        <name>dc1</name>
        <retentionindays>2</retentionindays>
      </source>
      <source>
        <name>dc2</name>
        <retentionindays>2</retentionindays>
      </source>
    </sources>
    <destinations>
      <destination>
        <name>dc1</name>
        <retentionindays>1</retentionindays>
        <primary>true</primary>
      </destination>
      <destination>
        <name>dc2</name>
        <retentionindays>1</retentionindays>
      </destination>
    </destinations>
  </stream>
</streams>

<clusters>
  <cluster name="dc1" hdfsurl="hdfs://dc1.bla.blacom:54310/"
           jturl="dc1.bla.bla.com:54311">
  </cluster>

  <cluster name="dc2" hdfsurl="hdfs://dc2.namenode.bla.com:54310"
           jturl="dc2.jobtracker.bla.com:54311">
  </cluster>
</clusters>
</databus>

Stream produced in three D.C.‘s. Need primary stream in one and mirror in others

I want to run databus in three data centers DC1, DC2, DC3. A stream name “example_stream” is being produced in each data centers. Need merged stream view in DC1 and mirrors in DC2 and DC3. Also need local streams in each data center.

<databus>
 <defaults>
   <rootdir>/databus</rootdir>
 </defaults>
 <streams>
   <stream name="example_stream">
     <sources>
       <source>
         <name>dc1</name>
         <retentionindays>2</retentionindays>
       </source>
       <source>
         <name>dc2</name>
         <retentionindays>2</retentionindays>
       </source>
       <source>
         <name>dc3</name>
         <retentionindays>2</retentionindays>
       </source>
     </sources>
     <destinations>
       <destination>
         <name>dc1</name>
         <retentionindays>1</retentionindays>
         <primary>true</primary>
       </destination>
       <destination>
         <name>dc2</name>
         <retentionindays>1</retentionindays>
       </destination>
       <destination>
         <name>dc3</name>
         <retentionindays>1</retentionindays>
       </destination>
     </destinations>
   </stream>
 </streams>

 <clusters>
   <cluster name="dc1" hdfsurl="hdfs://dc1.bla.blacom:54310/"
            jturl="dc1.bla.bla.com:54311">
   </cluster>

I want to use the same cluster to run databus to simulate running in three datacenters

I want to run databus in three data centers DC1, DC2, DC3. A stream name “example_stream” is being produced in each data centers. Need merged stream view in DC1 and mirrors in DC2 and DC3. Also need local streams in each data center.

<databus>
 <defaults>
 </defaults>
 <streams>
   <stream name="example_stream">
     <sources>
       <source>
         <name>dc1</name>
         <retentionindays>2</retentionindays>
       </source>
       <source>
         <name>dc2</name>
         <retentionindays>2</retentionindays>
       </source>
       <source>
         <name>dc3</name>
         <retentionindays>2</retentionindays>
       </source>
     </sources>
     <destinations>
       <destination>
         <name>dc1</name>
         <retentionindays>1</retentionindays>
         <primary>true</primary>
       </destination>
       <destination>
         <name>dc2</name>
         <retentionindays>1</retentionindays>
       </destination>
       <destination>
         <name>dc3</name>
         <retentionindays>1</retentionindays>
       </destination>
     </destinations>
   </stream>
 </streams>

 <clusters>
   <cluster name="dc1" hdfsurl="hdfs://dc1.bla.blacom:54310/"
            jturl="dc1.bla.bla.com:54311">
     <rootdir>/databus_dc1</rootdir>
   </cluster>

   <cluster name="dc2" hdfsurl="hdfs://dc2.namenode.bla.com:54310"
            jturl="dc2.jobtracker.bla.com:54311">
     <rootdir>/databus_dc2</rootdir>
   </cluster>

   <cluster name="dc3" hdfsurl="hdfs://dc3.namenode.bla.com:54310"
            jturl="dc3.jobtracker.bla.com:54311">
     <rootdir>/databus_dc3</rootdir>
   </cluster>
 </clusters>
 </databus>

Note: rootDir is defined as a separate directory for each cluster/D.C. in the above config