How to Use Data Shops: A Step-by-Step Guide to Create Your Custom Data Marketplace
by Brenna Dilger, on December 8, 2021
So you want to start a lucrative data business? You’ve come to the right place!
Launching an e-commerce data business is an exciting way to create a new source of revenue and share your valuable data assets. However, launching your data monetization initiative is not as cut and dried as simply uploading your data onto the internet and then leaving it to generate profit on its own. A successful data business requires the right tools and strategy to give you the best ROI. That's where Data Shops comes in!
In this walk-through guide on how to build a successful ecommerce data business, I’ll illustrate the best practices for launching your new initiative and I’ll be building my own Data Shop with you as a helpful example.
Let's get started!
Step 1: Determine What Data You Want to Sell and Who Your Buyers Are
Mapping and Assessing Your Data
You are the expert on your own data. When creating your unique data monetization strategy, you should be able to pinpoint the data that is valuable to other organizations. Determine what types of marketable data assets you have and assess their value. Begin with a thorough audit and appraisal of the data you have gathered within your organization. This includes data on processes, transactions, demographics, app usage, geolocation, and any other data that your company accumulates.
Here are some sample questions you should ask when taking inventory of your data:
- Do you have a comprehensive list of data that you think can be monetized?
- Is the data considered consumer data or personal data?
- How are you collecting information?
- Which systems do those datasets live in?
- What is the format and schema of the data in those systems?
- Is it easy to get the data out of those systems and move them to other platforms?
- How do you approach the value of data? Do you have an idea how much the data you’re looking to monetize is worth to you?
- Do any of the datasets present regulatory risks or challenges?
- How do you think your customers will use the data?
- Will the data be used for marketing purposes?
Download this checklist to help you take inventory of your data in a strategic way.
For my Data Shop, I have decided that I want to sell data on chocolate bar ratings from around the world. I’ve acquired a publicly available dataset that contains information on each chocolate bar’s rating, regional origin, percentage of cocoa, the variety of chocolate beans and other ingredients used, and where the beans were grown.
I will be creating this data shop for illustrative purposes and setting the prices of my data products to zero since the dataset I am using is a publicly available dataset.
Envisioning Your Audience
Determine which external parties might be interested in purchasing and utilizing your data so that you can craft your data monetization strategy around your potential audience. Look at how your company uses your data internally and imagine how those same techniques might be applied in other organizations. Estimate how much your data is worth to your company and determine if that data valuation would apply externally as well. Ask yourself what types of companies fit the profile of buyers who would consider your data a valuable resource.
Think about these questions:
- Who would find your data valuable?
- Who would be willing to pay for it?
- In what way is your data valuable? Will it help other organizations cut costs, streamline operations, increase sales, gain insights on customers, or all of the above?
- Would organizations outside of your industry benefit from the data you have?
Identifying an audience will help you later in your data monetization strategy when you are organizing your data and deciding how to market that data to potential buyers.
For example, I can envision that the potential buyers for my chocolate bar data might be chocolatiers, candy shops, or grocery stores.
Reviewing Policies, Laws & Regulations
It’s important to be aware of the different types of laws, regulations, and other legal issues that surround data monetization. You should be fully aware of all the rules surrounding data collection and data monetization in order to be sure that the data you are sharing is all above board. These rules have been put in place to make sure that citizens have protections when companies are collecting information about them and force companies to be more diligent when it comes to their data collection practices. Ultimately, they are designed to create a healthier more sustainable ecosystem while protecting the rights of consumers.
Step 2: Upload & Optimize Your Raw Data
Now that you have determined what data you want to monetize and who you want to market that data to, the next step is to ingest your data into Narrative’s data commerce platform using the first app in Data Shop’s suite of apps, Dataset Manager.
Using Dataset Manager, you’ll be able to create a dataset or multiple datasets reflecting exactly how your data is stored in your native system. Any kind of raw data, in any schema, can be taken exactly as it is stored in your system and pushed into the Narrative data commerce platform using Dataset Manager. All you have to do is pull the data from wherever it is stored, upload it into our system, and input the information that will describe your data and make your data discoverable to buyers.
Creating a New Dataset
To get started, install and launch Dataset Manager. Then click New Dataset.
This will take you to Set Dataset Info. On this page, you will be entering information that will help buyers find your dataset when they are searching. It’s important in this step to provide enough descriptive information and tags that will help with SEO.
Make sure you select Upload File at the top of the page to ensure that Dataset Manager will infer all the fields and primitive types for you. This way, you don’t have to do any meticulous manual work to get your data ingested into our platform!
Describing Your Dataset
You’ll need to provide descriptive information about your dataset and the fields it contains to identify and differentiate your dataset and fields.
You must enter a Name, Description, and Tags for your dataset. Make sure that you are labeling and describing each field in a way that makes them easily searchable and findable.
- Name: Enter the name of your dataset. Buyers will never actually interact with this name, but the name you give each dataset will help you find and differentiate each of the datasets you upload on the seller side. I’ve titled my dataset “Chocolate Bar Ratings From Around the World.”
- Description: Enter a good description of your dataset that will help you keep track of what is contained within the dataset. For my description, I’ve given a quick but descriptive summary of what type of data I have to offer on each chocolate bar: “Ratings of chocolate bars, based on their regional origin, percentage of cocoa, the variety of chocolate bean and ingredients used, and where the beans were grown.”
- Tags: Enter tags that will categorize your data when you are searching through all of the datasets that you will upload in the future. I’ve entered tags such as: chocolate, chocolate bars, cocoa, candy, etc. Be sure to hit the “return” key after you type each tag to save it.
After providing a description of the dataset, we need to let Dataset Manager know how our data is structured, also known as the schema. You can enter this information manually, but Dataset Manager is also able to infer the schema on its own just by taking a look at a small snippet of your dataset. It’s super easy to do so we’ll be doing it that way.
If your dataset is bigger than 10 megabytes, just copy the first few hundred rows of data into a new CSV or JSON LD file. Then you simply drag and drop your chosen file and click Generate Fields to successfully upload your sample file and move on to defining your data.
Defining Your Data
You will now be taken to Define Settings. In this section, you will define the settings of the files containing your data. Dataset Manager has inferred the schema, so you will just have to review this section to make sure your data was inferred correctly and make any changes if you spot a discrepancy. Though this page should be easy to glance at and click through, you should review all of the below sections to ensure everything is correct:
- File Type: This is the way your data is stored and tells the Data Streaming Platform how to parse your raw data. Do not change this as the system will never incorrectly infer your file type.
- Header: Keep this on if the first row of your file contains field names. Switch this off if it does not contain field names.
- Delimiter: Choose from the dropdown menu which character separates the data in your file into distinct fields.
- Escape Character: Enter the single character used for escaping quotes inside an already queued value.
- Quote: Enter the single character used for escaping quoted values where the separator can be part of the value.
- Write Mode: This field informs how to treat new data uploaded to your dataset.
- Select Incremental Updates if new data should be added to the end of your dataset.
- Select Complete File Updates if new data should replace your entire dataset.
Once you have double-checked everything in Define Settings, you can move on to the Define Fields section, which will display an automated list of your dataset’s fields.
Dataset Manager’s file inference guesses the field names and primitive types of each dataset for you, but you should still check each field for accuracy and be sure to provide a description for each field. Adding a description for each field will further increase the discoverability of your data products and help you and your buyers keep organized.
In order to edit or delete a field, click on the ellipses in the right-hand corner of the field and choose Edit or Delete. In Edit, A dropdown menu will appear. For each field, make sure these sections are filled out and accurate:
- Name: This is the title that identifies the field. A title will be generated for you, but you can amend the title to be more descriptive of each field. The title should not contain spaces or non-alphanumeric characters other than underscores (i.e. cocoa_percent).
- Description: Make sure you provide information about each field to your buyers as it is important for discoverability and distinction. For my chocolate bar brand field, I gave a quick sentence clarifying that this field contains the percentage of cocoa in the chocolate bar being rated.
- Primitive Type: Make sure that each primitive type is correct as this controls the filters available to you in other Data Shops apps.
- Boolean-type columns will only recognize "true" and "false" values (case-insensitive, e.g. "TRUE" and "False" are also ok). Any other value will be ingested as "null.”
- Choose a string type for columns containing text, such as identifiers, even if the identifiers consist of numbers.
- Choose a long type for integers, e.g. whole numbers and Unix timestamps.
- Choose a double type for numbers with decimals, e.g. monetary values and coordinates.
- Choose a timestamptz type for timestamps like "4/7/2021 11:33:07". (Choose a long type for Unix timestamps like "1606694399999".)
- Required: Switch this on if the field must contain a value for a record to be added to your dataset. If this is switched on, any record without a value in the number field is not going to be ingested.
- Sellable: Switch this on if you want the field to be made available to buyers. If you turn this field off, you’ll still be able to filter by this field to create data products, but buyers will not receive any of the information contained in this field whenever they buy a data product from your shop
- Approximate Cardinality (Optional): This is the estimated unique values in your field. Providing the estimated cardinality of field values will allow your dataset to be stored and processed faster.
- Field Values (Optional): You have the option of either setting a list of Predefined Values for your field or leaving it open to accepting Any values.
- Any: All values in this field are acceptable for a record to be ingested.
- Predefined Values: Only values that you enter will be considered valid. Records with any other values will not be ingested.
Once you have made sure each field is filled out and accurate, you will set the Primary Column. This is the main defining field in your dataset and describes the field that buyers are going to be filtering most often. I have set mine to “company” because it is the main differentiator for each chocolate bar that was rated (buyers will be interested in filtering through which companies had higher rated chocolate bars).
Now you can continue on to activate your data set!
Activating Your Dataset
You will then be taken to the Review and Activate page. This is your final chance to review your dataset before activating it. You will not be able to make changes once your dataset is active.
If you want to go back to another section to make any changes, simply click the right-hand arrow in the corner of the section you want to revisit.
Once you have reviewed all of your information and ensured that everything is accurate, you are ready to activate your dataset!
Congratulations! You have now created your first dataset in Dataset Manager!
You can search through your data lake with the Search bar in the left-hand corner of the page and choose how you want your data lake organized with the dropdown menu in the right-hand corner of the page.
Ingesting Your Data
The next step is to add data to your dataset, a process called "ingestion.”
Once data is ingested into your dataset, you can use Seller Studio's customizable filters to create, package, and sell different data products on your Data Shop and/or the Data Streams Marketplace.
To ingest your data, you’ll have to set up a managed bucket and use your managed bucket to upload files for ingestion to your datasets. Information on how to complete both of these processes is available in Narrative's Knowledge Base.
Step 3: Create Buyable Data Products
Now that your data has been ingested, it’s time to create some products and give your buyers access to the data you want to sell.
All of the data you have ingested can be divided and packaged into products that you think your customers would like to buy individually - these are called data streams. Each data stream you create is a specific set of attributes that you pull from your original dataset and make available to your buyers.
For instance, some of my data could be organized into products like: Women’s Rating for Chocolate Bars, Argentinian Chocolate Bar Ratings, American Men Aged 18-34 Ratings for Chocolate Bars, etc. You can be as specific as you would like to be and create as many products as you want.
To start creating products, we’ll be using the Seller Studio app.
You can do two things in Seller Studio:
- Create prepackaged products, or data streams, that can be added to a cart for an eCommerce-style of buying.
- Manage custom access to your data by creating access rules for enterprise buyers to create their own data streams (and potentially buy from more than one seller at a time).
Creating Data Streams
To create a pre-packaged product of your own, click New Data Stream. A list will appear of all the datasets you’ve already set up with Dataset Manager. Select which dataset you would like to turn into a product. Most sellers can create tens or hundreds of data streams from just one dataset.
Once you have selected a dataset, you’ll be taken to Choose Columns. Here, you can choose exactly which fields will be delivered to someone who is buying your dataset and make fields within your dataset filterable.
Specifying Deliverability and Filterability
Specify the fields you want to be delivered to your buyers by keeping Delivered switched on and switching it off for any fields that you don’t want buyers to receive. A field that is not deliverable will not be sent to a buyer.
If you want to specify a subset of your dataset that is going to be included in the data stream, you can choose to make each field filterable by keeping Filterable switched on. If you want to leave something unbounded and include all the records no matter what the field contains, turn off the Filterable option.
I’ve chosen to make all of my fields deliverable and filterable to give my buyers the most options and myself the most customizability.
Continue to Apply Filters.
Setting Smart Filters Based on Your Dataset
Depending on the dataset you created in Dataset Manager, our user interface will generate smart filters based on the primitive type of each field.
You’ll be able to select the data you want to include in each field by choosing from Include all values, Include if present, or Custom. Using these filters, you’ll be able to make each field as specific as you want. Include if present will make sure a value exists in that row of data in order to include it in your product. Custom allows you to include or exclude any values you choose to list.
For instance, if I want to create a data product that only features chocolate bar rating data from USA or Canada based companies, I would be sure to specify that in my “company_location” field:
I could also specify that I only want chocolate bar ratings for chocolate bars that contain between fifty percent cocoa and seventy-five percent cocoa:
You can create a data product based on any attributes you choose to modify or you can leave everything exactly as it is and allow your buyers to access all of your dataset’s data in one bundle.
Once you’ve made your modifications and organized the data into a product you’re happy with, continue on to Set Offers.
Making Your Data Streams Available to Buyers
You have the option to make your data stream available for sale in the Narrative Data Streams Marketplace as well as in your own Data Shop. To increase visibility and maximize sales, I recommend selecting both Sell on the Data Streams Marketplace and Sell on my Data Shop. This way, your data streams will be available to all potential buyers and will be found much more frequently.
If for whatever reason you don’t want your data streams available in one or the other, simply switch that option off.
Pricing Your Data Streams
You’ll set a price for every 1,000 records of raw data by inputting your preferred price in Cost Per Thousand or adjusting the price scale.
Pricing your data is an important step. You can choose to make your price competitive or sell your data at a premium depending on what type of data you are offering. Decide the market price of your data products based on your organization’s valuation of your data. For further inspiration, take a look at your competitors’ pricing on their data products and compare them to your own products.
How exclusive is your data? How valuable is your data compared to other data products related to your industry and how much do you think other organizations would pay to have access to your particular insights?
I’ve decided to set my price to a fair $3.00 CPT based on my own analysis of the type of data I have to offer.
Licensing Your Data Streams
This step is taken care of for you. Every platform participant agrees to the Narrative standard licensing terms when they register. Under the Narrative Marketplace Data Purchase Agreement, buyers are given unlimited use of the licensed data within the specified license expiration period.
Continue to Description.
Practicing Good SEO For Your Data Products
You can set merchandising information and provide a product image to appeal to buyers.
The more detail you provide (description, tags, name, etc.), the easier it is for Google to index your products and for customers to find them and understand what they're getting. Be clear, descriptive, and specific in this section in order to give your data products the best shot at being discovered and purchased.
- Name: Decide what you want your data stream to be called. The name of the data stream will be searchable and displayed to customers, so make sure it is specific and clear. I decided to create a data product that has only ratings and data on chocolate bars that were produced by companies based in the USA and Canada, so I titled it to specify that: Chocolate Bar Ratings - USA & Canada Based Chocolate Producers
- Slug: This is the permalink for the data stream product page, which can positively impact SEO. Choose a unique and distinguishable URL where your data product will live on the internet. I’ve chosen “usa-canada-chocolate-bar-ratings” in order to make my URL specific and easily identifiable.
- Description: This information will explain your dataset to interested buyers and will help buyers discover your dataset on search engines. This is your opportunity to explain to buyers why they care about your data stream, what it can be used for, how the data was collected, and any other information you want to share to make your data discoverable and entice buyers. Make sure this section explains exactly what kind of data your product contains and highlights what is valuable or unique about the product.
- Image: To make your product look extra purchasable, provide a product image to showcase the data stream. Products that include a relevant image tend to be more exciting and come across as more professional to potential buyers and make your products easily browsable in your storefront.
- Category: By positioning your product in a relevant category, you are making it extra easy for buyers to find your product. Any buyers searching specifically under the umbrella category you choose will have an especially high chance of finding your data stream. Be sure to make the category specific and relevant.
- Tags: Enter tags that will categorize your data and make it discoverable to buyers. Try to imagine what terms your buyers might be searching for if they wanted to find the data you have to offer. The more tags you enter, the better! I’ve entered tags such as: chocolate, candy, chocolate bars, usa chocolate bars, etc. Be sure to hit the “return” key after you type each tag to save it.
Click Continue to Review.
Activating Your Data Stream
Preview the product card as it'll appear on the Data Stream Marketplace and/or on your Data Shop. Make sure your product card is attractive, descriptive, categorized, and aptly priced.
If you’re happy with how your product appears to buyers and have optimized every possible factor, click Activate Data Stream. Your product will now be available in the Datastreams Marketplace.
In minutes we've created your first data stream!
Creating Access Rules
Setting access rules is how you will make your data available for buyers to add to custom subscriptions through our app, Buyer Studio. By setting access rules for your data streams, you’re opening up another channel with which buyers can find and buy the precise data you want to sell.
To get started, click Access Rules on the left-side menu on Seller Studio.
Select a dataset in the dropdown menu and scroll down to click Add Access Rule.
You will need to provide a Description, which identifies the access rule to buyers. For mine, I labeled it “Open Access - USA and Canada Chocolate Bar Ratings” This title indicates that everyone on the marketplace has access to my dataset and also describes what data they will be receiving.
In the Cost Per Thousand section, you’ll price your data similarly to how you priced it when you made your original data stream.
Setting Up Constraints
To limit the data you want to make available to buyers building a subscription, you will set up Constraints, which will filter which records you want to make available in your sellable dataset.
Click Add Constraint to select from a dropdown menu of attributes and options to make those attributes NULL or include them in the dataset.
Choosing Your Buyers
You have the ability to adjust the discoverability of your data products. You can either allow all buyers to have access to your data or make your products exclusive to certain buyers. To get the maximal amount of revenue, you’ll want to make your data streams available to everyone in the marketplace. Click the Buyers drop down menu and select Sell to all Buyers.
You also have the option of selecting Custom. With that option, you can choose to Include or Exclude any other users within the marketplace. You might choose to do this if you have a direct partnership with another user or simply don’t want to provide specific users (such as competitors) with your data.
Now that you have created a data stream and made it accessible for custom subscriptions, you can go to Shop Builder to set up a beautiful ecommerce storefront!
Step 4: Build an Attractive Branded Storefront
Now you can set up an aesthetically appealing ecommerce storefront and emphasize your unique brand in order to virtually shelve your data products and stand out from the crowd. All of this is possible with Shop Builder. Make sure you have the app downloaded to get started.
Once you’ve opened Shop Builder in your browser, select Data Shop Settings and then click Create a New Data Shop.
Defining Your Shop’s Brand & Voice
Before you begin setting up your Data Shop customizations, take a moment to decide what logo, colors, tone, and images you want to represent your data brand. Think about how you want to convey your data business brand and the products that you’re offering.
Do you want your shop to be fun and quirky with cute pictures and silly titles and descriptions or do you want to take a more professional tone? Do you want your ecommerce storefront to be colorful and bright or monochrome and subdued?
If you want your data shop brand to match your organization’s brand and voice, be sure to align the design and written content of your data shop with the tone and style of your unique brand.
Go to Shop Settings and click Create a New Data Shop.
Creating and Optimizing Your Domain
First, register a Domain Name. This will be the name of your data store website so make sure that it is simple and representative of your brand and data products. Since my brand is all about chocolate and chocolate-centered data, I decided to title my data shop with my name and industry. I’ve chosen the domain name: brennas-chocolate-datashop.com.
Also be sure to point your domain to Narrative's DNS server.
Now, you’ll need to optimize your data store by using best SEO practices to get your data products found. Be sure to follow these tips for each section:
- Google Analytics: Set a Google Analytics tag for your shop. This tag will help your data products pop up in Google search results. Think about what your buyers might be searching for if they wanted to find your data and enter the most precise and popular term or phrase. For instance, I set my tag to “chocolate data.”
- Google Tag Manager: Set your Google Tag Manager tracking tag to the same phrase or term that you entered in the above field. Google Tag Manager will help you keep track of when, where, and by who your data shop is being visited and purchased from.
- Shop Name: Give your shop a unique name that will represent your brand. The shop’s name will appear on the web browser’s title bar and will be the first thing your buyers see. I have chosen to name my shop “Brenna’s Chocolate Datashop.”
- Meta Description: This is the description of your Data Shop as it appears to a search engine. Make sure that it describes your data in a succinct manner.
- Meta Tags: These tags describe your website to search engines. For my chocolate data shop, I’ll enter tags such as “chocolate data,” “chocolate bar data,” etc. Be sure to use the return key to save each tag.
- Featured Search Terms: These are suggested search terms that will appear as buttons below the search bar on the Home page. Use the return key to save a search term.
Setting Your Visibility
You can choose to list your shop on the Narrative Data Streams Marketplace or require visitors to login and/or register before entering your Data Shop. To make your shop the most discoverable, you should choose the Data Streams Marketplace option.
Customizing Your Shop’s Appearance
Enter your Site Logo’s image URL. This is simply the logo of your brand and should be uploaded so that your organization and its data assets are easily identifiable with a glance. Having a nice logo also makes your shop look ultra professional and polished. (Note that the logo height is scaled to 28 pixels. The width of the logo will change based on the aspect ratio of the source image. An image with a transparent background is recommended.)
Enter the Site Logo Alt Text, which is the text that will pop up when a user moves their cursor over the logo image. This should be no more than a few words to a sentence long and should give a quick written identifier of your brand. It could be your company’s name, slogan, or a direct description of who your brand is and what products you’re providing.
Next, provide a Placeholder Image URL. This image will be used next to your data products if you forget or choose not to upload an image when you post the data product for sale.
Lastly, provide a Favicon Icon URL. This icon will appear next to the page title in the browser's title bar. This could simply be your logo image or you could get creative with it. (Note that this icon will be scaled to 16 x 16 pixels.)
Sending Exclusive Invites
You’ve created your data shop! It’s now open for business and can be filled with as many data products as you’d like.
You can also notify your customers with an email invite that you have set up your first Data Shop!
Delivering & Managing Purchases
Once your data has been reviewed, organized into marketable products, and displayed in an attractive ecommerce shop front, you can sit back and relax. When a buyer makes a purchase from your data shop, Narrative will deliver the purchased data automatically to the buyer, so you can simply check in on your data business at your leisure without having to monitor purchases.