BackBone MLS / XML syndication [big data parallel processing] platform

BackBone is 4Property’s specialist IT services department for big data classified advertisment processing.

“Our Data Backplane Integration Architecture is designed for high performance and flexibility. Leveraging Confluent^TMTechnology, it allows for seamless data transfer between diverse sources and sinks. This near real-time data synchronization, with only about a 500 msec delay from source to sink, enhances the responsiveness and reliability of our services.”

Infrastructure Deployment – Modern Cloud based Infrastructure as Code approach

We deploy our architecture on a robust Kubernetes infrastructure, integrated with Istio Service Mesh. Istio enhances our microservices by providing real-time monitoring and resiliency. This robust deployment contributes significantly to achieving our target Service Level Agreement (SLA) for  99.94% uptime, ensuring our services are available when our customers need them.”

All our infrastructure is defined using Code Based Configuration capabilities provided by Kubernetes that is backed by Github repositories and tooling that helps us to address service failures in near real time. 

Furthermore, we have Slack based Health Notifications and Escalation Management so we get to know about any anomaly and take preventive actions.

KSQLDB with KSQL for responsive Feed Turnaround

• “We utilize KSQLDB, an event streaming database purpose-built for stream processing applications, to define and process business logic on the pipelines. Working in conjunction with Kafka Connect Infrastructure, we are able to source data from major RDBMS and Document databases, as long as the necessary network access is provided. Furthermore, our infrastructure supports change data capture drivers for real-time processing of record updates—this enables true event-based, stream processing.

• All custom business logic for pipelines are created as KSQL migrations which can be propagated through our build system automatically. We do aggressive testing of the same using our CI/CD infrastructure.. 

• The world of business has to accommodate for changes in their respective domains. We can update i.e. migrate pipelines without bringing down our services or suffering from loss of data.

Connecting to Your Systems – Kafka Connect

• Kafka Connect also allows us to consume REST API’s for pulling data through a scheduled polled mechanism, providing flexibility to accommodate various data sources and meet specific business requirements. We can also develop custom connectors for customer-specific service APIs, offering personalized solutions to cater to unique business needs.
• Our infrastructure is designed to support both real-time and batched modes for data fetching. For batched polling mode, we provide support for incremental updates if timestamp data is available on the data records. This timestamp data should be based on ISO 8601 or Unix epoch time format.
• Additionally, when dealing with external APIs, we’ve implemented strategies to handle potential load constraints. For instance, we can constrain the number of updates and perform rolling micro-batched updates, thus ensuring we maintain high performance without overwhelming the API.
• All these features of our Kafka Connect infrastructure contribute to efficient real-time data processing, which greatly enhances our service availability and real-time data synchronization.”

RESTful API & WebSocket based Services – Have one, No Problem!

“To make data integration easy for our customers, we’ve can wrap RESTful API services as sink or source connectors to our Data Backplane. This allows your systems to access our data quickly and efficiently, enhancing overall user experience and service availability.”

“For more complex use cases, we can deploy custom microservices that push data on change to your systems through Websocket protocol.”

“These microservices are designed in consultation with you technical leads to match your technical and security expectations”

Sigma Data Processing Architecture – Our Answer for Big Data ETL

“Our Sigma Data Processing Architecture leverages the power of event sourcing and streaming data processing. This offers benefits such as the ability to reconstruct past states, perform complex events tracking, and ensures a highly reliable and efficient data processing system.”

“Also we are using modern distributed architecture to ensure that there is always available compute power to processes millions of records if needed so. These systems are dynamically provisioned and used to keep operating costs down. Our intention is to pass on these benefits to customers through tiered pricing as your business grows with us.”

Monitoring & Load based Availability- Around the clock

“To ensure optimal performance and service availability, we use Opentelemetry technologies in conjunction with Istio Service Mesh for comprehensive message monitoring. This fusion of technologies allows for real-time tracking, quick identification, and resolution of issues, leading to enhanced system reliability and uptime.

Our systems are capable to Scale On Demand based on rules that are triggered if shortage of resources are felt during times of intensive operation. 

These scaling functions operate without human intervention and is based on Application Metrics that are collected through our Telemetry Systems”

“In conclusion, our Data Backplane Integration architecture and Feed services, leveraging Confluent Technology, Kubernetes, Istio Service Mesh, KSQLDB, Kafka Connect Infrastructure, and Sigma Data Processing Architecture, is designed to provide reliable, efficient, and resilient services. We’re committed to maintaining near real-time data synchronization and processing, as well as a high level of service availability, offering our customers the best possible experience.”

“We also can work closely with your development leads if technical requirements arise during the delivery of a customized solution.”

“We are also continuously improving this infrastructure. As part of our technical roadmap we working to provide additional services such as ability to trace your data directly through a portal. This portal will also help your service managers to query and trace data packets as they pass through our system.” 

Back story

Some years ago, the founders of 4property build a portal and build out a datafeed solution to process the hundreds of datafeeds from multiple CRM’s. Then came the queries about whether a listing had pubished or not and it became clean that a new approach was needed so we started development on what we now call the Dataplane. We identified a serious cost in supporting datafeeds and we sought to provide a solution:

Real Estate Agents, in particular, were eager to showcase their properties on various websites to tap into the growing number of property seekers utilizing the internet for their searches.

The obstacle was the time-consuming process of listing each property individually on each site. Automation was required as agents will not publish even if it is offered for free to list.

As a result, agents often limited their listings to a handful of sites, and updates to these listings were infrequent.

The solution was an automatic ‘feed’ of the agent’s property listings – so that when a property was added to his own website, it automatically got distributed (or fed) to the chosen sites for syndication.

Dataplane engineered the systems necessary to ‘harvest’ the property listings from the agent’s CRM, agents website, or from the portals api and convert that data into a format that other websites or systems could easily import, and then ‘feed’ it to them.

This innovation allowed real estate agents to have their properties ‘syndicated’ broadly across multiple websites with just a click.

What Dataplane does –

By tailoring our software to the layout of the client’s website, we create a data extraction robot to ‘crawl’ the site (similarly to how Google indexes a site) and replicate the listings into our database.

The replicated data then moves to our feed generator, which reformats the data into various formats that other websites can interpret and import. We generate multiple versions of the feed, ensuring compatibility with 97% of the world’s portals. Once the ‘feed’ is prepared, it’s ready to be sent to your chosen recipients. To maintain accurracy, our ‘robot’ revisits the data source (your site or export file) daily to capture your most recent updates, thus allowing feed recipients to automatically refresh their systems with your latest listings, deletions, and amendments each day.

Benefits to you:

Massive time savings – the broader the distribution of your feed, the more significant the time saved, translating directly into financial savings. We streamline everything to make the process as automatic as possible. You simply decide the recipients, and we handle the identification, selection, and sending of the feed.

More Services: Hosting, Registration & DNS Management, SSL Certificates, IT Consultancy

Many of today’s organisations are ever so reliant on their software applications and network infrastructure to ensure business continuity. As a consequence, many businesses are spending more time operating their Information Communication Technology (ICT) infrastructure, rather than focusing on their core business functions.

When 4Property manages your data infrastructure, you will be able to focus on running your business without sacrificing quality or service, and enable your business to become more profitable and less stressful.

Preventive
Maintenance

We monitor your systems and security 24/7 to ensure that we are already solving problems before you even know you have them.

Remote
Support

A complete IT department at your disposal providing experienced remote and on-site* support whenever you need it.

* We charge extra for on-site support.

Full Network
Maintenance

We manage and maintain your entire network including workstations, servers, wifi, and internet connections.

Server & Network
Support and Management

Too many businesses let their servers, data, and networks go unmanaged for weeks or months at a time. Don’t expose your central IT infrastructure to risk by ignoring warning signs or failing to perform maintenance and security tasks. Let us take care of your systems to ensure business productivity.

Strategic
Technology Planning

Research and implement new solutions to meet changing business needs. We build a custom IT roadmap for your business, then meet with you on a regular basis to ensure that your systems are both up to date and secure.

Technology
Procurement

Through our wide reaching channel partners we are the ideal partner when it comes to purchasing equipment, software and services.

4Property are
Official Google Resellers

A set of intelligent apps including Gmail, Docs, Drive and Calendar to connect the people in your company, no matter where in the world they are.

Create documents, spreadsheets and presentations from anywhere. Share them with teammates and work together on the same file, at the same time.

Enjoy a consistent experience from your computer, tablet or phone. Draft a proposal in Docs at the office, review it on the train, then make final edits from your phone right before the meeting. Get more out of your workday.

One Account
for Everything

Accessing your work is easy with one login for everything you do.

Administrative controls offer two-step verification to enhance security for the whole company.

With email, chat and video meetings you’re always up to date. Meet face-to-face with co-workers or customers no matter where you are.

Keep Data
Secure and Accessible

Store all your work in one place and quickly find what you need with Google-powered search. Access it anytime and if you decide to share, you control how to do it.

With G Suite, comprehensive security is built-in at every layer, from reliable infrastructure to advanced detection systems. That’s why some of the world’s most discriminating industries and customers have chosen G Suite.

Hosting

Is what makes your site visible on the web. We offer fast, reliable and secure solution for every need – from a basic blog to high-powered site. Designer? Developer? We’ve got you covered too.

SSL Certificates

For your business to succeed, customers need to trust that you’ll protect them from viruses, hackers and identity thieves. Count on our security products to keep your website secure, your visitors safe and your business growing.

Domain & DNS Zones

You can’t have a website without a domain name. Like a street address that tells people where you live, a domain helps customers drive directly to your website. We can help you register, control and manage it for you.

IT Consultancy

If your business requires a simple set of upgrades, a redesign of your current systems, or a brand new system implementation, 4Property can help. 4Property will perform an analysis of your existing infrastructure and business requirements.

The results of our analysis will allow us to design a cost effective system to handle your business needs.