0

I have this following JSON data;

data=[
 {
        _id: "5b377db0c97f730014b6eb12",
        title: "Integrated Compute Platform - Operations Lead",
        applylink:
          "https://www.techgig.com/jobs/Integrated-Compute-Platform-Operations-Lead/60221",
        jd: "",
        companyname: "JP Morgan Chase Bank",
        location: "Hyderabad/Secunderabad",
        experience: "5-7 yrs",
        salary: "",
        type: "",
        skills: "Cisco",
        startdate: "",
        enddate: "",
        created: "",
        source: "techgig",
        timestamp: 1530363306.1030896,
        __v: 0
      },
      {
        _id: "5b377db0c97f730014b6eb13",
        title: "angular-ui/ux",
        applylink: "https://www.techgig.com/jobs/angular-ui-ux/60213",
        jd: "",
        companyname: "Capgemini",
        location: "Pune",
        experience: "6-9 yrs",
        salary: "",
        type: "",
        skills: "UI / UX",
        startdate: "",
        enddate: "",
        created: "",
        source: "techgig",
        timestamp: 1530363306.1030896,
        __v: 0
      },
      {
        _id: "5b377db0c97f730014b6eb14",
        title: "BCM - Big Data CoE Lead",
        applylink: "https://www.techgig.com/jobs/BCM-Big-Data-CoE-Lead/60226",
        jd: "",
        companyname: "Capgemini",
        location: "Pune",
        experience: "17-20 yrs",
        salary: "",
        type: "",
        skills: "Big Data",
        startdate: "",
        enddate: "",
        created: "",
        source: "techgig",
        timestamp: 1530363306.1030896,
        __v: 0
      },
      {
        _id: "5b377db0c97f730014b6eb15",
        title: "Staff QA Engineer, Open VisaNet",
        applylink:
          "https://www.techgig.com/jobs/Staff-QA-Engineer-Open-VisaNet/60218",
        jd: "",
        companyname: "Visa",
        location: "Bengaluru/Bangalore",
        experience: "7-12 yrs",
        salary: "",
        type: "",
        skills: "Erlang",
        startdate: "",
        enddate: "",
        created: "",
        source: "techgig",
        timestamp: 1530363306.1030896,
        __v: 0
      },
      {
        _id: "5b377db0c97f730014b6eb16",
        title: "Hadoop - Big Data Developer",
        applylink:
          "https://www.techgig.com/jobs/Hadoop-Big-Data-Developer/60225",
        jd: "",
        companyname: "Morgan Stanley",
        location: "Mumbai",
        experience: "4-7 yrs",
        salary: "",
        type: "",
        skills: "Big Data",
        startdate: "",
        enddate: "",
        created: "",
        source: "techgig",
        timestamp: 1530363306.1030896,
        __v: 0
      },
      {
        _id: "5b377db0c97f730014b6eb17",
        title: "Specialist UX/UI Designer",
        applylink:
          "https://www.techgig.com/jobs/Specialist-UX-UI-Designer/60215",
        jd: "",
        companyname: "Hewlett Packard",
        location: "Bengaluru/Bangalore",
        experience: "5-9 yrs",
        salary: "",
        type: "",
        skills: "UI / UX",
        startdate: "",
        enddate: "",
        created: "",
        source: "techgig",
        timestamp: 1530363306.1030896,
        __v: 0
      },
      {
        _id: "5b377db0c97f730014b6eb18",
        title: "Hadoop - Big Data Developer",
        applylink:
          "https://www.techgig.com/jobs/Hadoop-Big-Data-Developer/60225",
        jd: "",
        companyname: "Morgan Stanley",
        location: "Mumbai",
        experience: "4-7 yrs",
        salary: "",
        type: "",
        skills: "Big Data",
        startdate: "",
        enddate: "",
        created: "",
        source: "techgig",
        timestamp: 1530363306.1030896,
        __v: 0
      }]

Data is lot more than that, around 1.5 MB coming from a JSON file hosted online, this is just sample, I have to filter the jobs on the basis of location, skill, experience.

What I thought of to add everything to the state then preprocess the data in the 3 diff array with the following format

      {
        value: jobData.xxx,
        label: jobData.xxx
      }

Push the data in react-select, get from that and use a filter for the whole state and display the result in a nice UI.

The problem here is data is really huge, and no option to get chunks from backend, I have to use the full data at once.

My questions are:-

  • How to store the data skill, location and exp segregate in diff array without the duplicated elements, I tried with a map, duplicated element are coming. Iterating through the whole array again to check if it's not there would not be efficient?
  • Is there a better way you all propose to do it?

Thanks

Edit-1

So basically what i want 3 json object

 var skill = {
        value: jobData.skills,
        label: jobData.skills
      };
      var location = {
        value: jobData.location,
        label: jobData.location
      };
      var experience = {
        value: jobData.experience,
        label: jobData.experience
      };

pushed in three array:

 var skillList=[];
    var locationList=[];
    var experienceList=[];

Which will be inturn set in react state

Edit-2

This is the whole code:

import React from "react";
import Style from "./Landing.module.scss";
import JobImage from "./2663543.jpg";
import Select from "react-select";
class LandingPage extends React.Component {
  state = {
    data: [
    //the data mentiond above
    ],
    skills: [],
    location: [],
    experience: []
  };

  componentDidMount() {

    var skillList=[];
    var locationList=[];
    var experienceList=[];


    this.state.data.map(jobData => {
      var skill = {
        value: jobData.skills,
        label: jobData.skills
      };
      var location = {
        value: jobData.location,
        label: jobData.location
      };
      var experience = {
        value: jobData.experience,
        label: jobData.experience
      };
    });


  }

  render() {
    return (
      <>
        <div className={Style.bigHead}>
          <div className={Style.bigImage}>
            <img src={JobImage} alt="Job Image"></img>
          </div>
          <div className={Style.filters}>
            <Select
              isMulti
              name="location"
              options={this.state.location}
              className="basic-multi-select"
              classNamePrefix="select"
            />
             <Select
              isMulti
              name="experience"
              options={this.state.experience}
              className="basic-multi-select"
              classNamePrefix="select"
            />
             <Select
              isMulti
              name="skill"
              options={this.state.skills}
              className="basic-multi-select"
              classNamePrefix="select"
            />
          </div>
        </div>
      </>
    );
  }
}

export default LandingPage;
23
  • You can use Map Commented Sep 14, 2019 at 9:50
  • You can use Set Commented Sep 14, 2019 at 9:51
  • Agreed, used map, got the value in the array, but tere is duplites comming, how can i apply filter and reduce Commented Sep 14, 2019 at 9:51
  • Yes, i tht of, but set is not supported by react-select where i have to feed the data :-(, i have to again process that set into the react-select Commented Sep 14, 2019 at 9:52
  • 1
    given the data Array you've show, can you be more descriptive with what you need as an "output" - e.g., you have 4 distinct skills in those 7 records ... did you want something that has those 4 skills with references to the 7 records they came from, or just a list of 4 skills? Commented Sep 14, 2019 at 9:59

1 Answer 1

2

This is one way to do it

let skillsList = [...new Set(data.map(({skills}) => skills))].map(value => ({value, label:value}));
let locationList = [...new Set(data.map(({location}) => location))].map(value => ({value, label:value}));
let experienceList = [...new Set(data.map(({experience}) => experience))].map(value => ({value, label:value}));

let data=[{ _id: "5b377db0c97f730014b6eb12", title: "Integrated Compute Platform - Operations Lead", applylink: "https://www.techgig.com/jobs/Integrated-Compute-Platform-Operations-Lead/60221", jd: "", companyname: "JP Morgan Chase Bank", location: "Hyderabad/Secunderabad", experience: "5-7 yrs", salary: "", type: "", skills: "Cisco", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb13", title: "angular-ui/ux", applylink: "https://www.techgig.com/jobs/angular-ui-ux/60213", jd: "", companyname: "Capgemini", location: "Pune", experience: "6-9 yrs", salary: "", type: "", skills: "UI / UX", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb14", title: "BCM - Big Data CoE Lead", applylink: "https://www.techgig.com/jobs/BCM-Big-Data-CoE-Lead/60226", jd: "", companyname: "Capgemini", location: "Pune", experience: "17-20 yrs", salary: "", type: "", skills: "Big Data", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb15", title: "Staff QA Engineer, Open VisaNet", applylink: "https://www.techgig.com/jobs/Staff-QA-Engineer-Open-VisaNet/60218", jd: "", companyname: "Visa", location: "Bengaluru/Bangalore", experience: "7-12 yrs", salary: "", type: "", skills: "Erlang", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb16", title: "Hadoop - Big Data Developer", applylink: "https://www.techgig.com/jobs/Hadoop-Big-Data-Developer/60225", jd: "", companyname: "Morgan Stanley", location: "Mumbai", experience: "4-7 yrs", salary: "", type: "", skills: "Big Data", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb17", title: "Specialist UX/UI Designer", applylink: "https://www.techgig.com/jobs/Specialist-UX-UI-Designer/60215", jd: "", companyname: "Hewlett Packard", location: "Bengaluru/Bangalore", experience: "5-9 yrs", salary: "", type: "", skills: "UI / UX", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb18", title: "Hadoop - Big Data Developer", applylink: "https://www.techgig.com/jobs/Hadoop-Big-Data-Developer/60225", jd: "", companyname: "Morgan Stanley", location: "Mumbai", experience: "4-7 yrs", salary: "", type: "", skills: "Big Data", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }];

let skillsList = [...new Set(data.map(({skills}) => skills))].map(value => ({value, label:value}));
let locationList = [...new Set(data.map(({location}) => location))].map(value => ({value, label:value}));
let experienceList = [...new Set(data.map(({experience}) => experience))].map(value => ({value, label:value}));

console.log(skillsList);
console.log(locationList);
console.log(locationList);

Another way, which may or may not be more performant I don't know, but it iterates data once, not 3 times

let data=[{ _id: "5b377db0c97f730014b6eb12", title: "Integrated Compute Platform - Operations Lead", applylink: "https://www.techgig.com/jobs/Integrated-Compute-Platform-Operations-Lead/60221", jd: "", companyname: "JP Morgan Chase Bank", location: "Hyderabad/Secunderabad", experience: "5-7 yrs", salary: "", type: "", skills: "Cisco", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb13", title: "angular-ui/ux", applylink: "https://www.techgig.com/jobs/angular-ui-ux/60213", jd: "", companyname: "Capgemini", location: "Pune", experience: "6-9 yrs", salary: "", type: "", skills: "UI / UX", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb14", title: "BCM - Big Data CoE Lead", applylink: "https://www.techgig.com/jobs/BCM-Big-Data-CoE-Lead/60226", jd: "", companyname: "Capgemini", location: "Pune", experience: "17-20 yrs", salary: "", type: "", skills: "Big Data", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb15", title: "Staff QA Engineer, Open VisaNet", applylink: "https://www.techgig.com/jobs/Staff-QA-Engineer-Open-VisaNet/60218", jd: "", companyname: "Visa", location: "Bengaluru/Bangalore", experience: "7-12 yrs", salary: "", type: "", skills: "Erlang", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb16", title: "Hadoop - Big Data Developer", applylink: "https://www.techgig.com/jobs/Hadoop-Big-Data-Developer/60225", jd: "", companyname: "Morgan Stanley", location: "Mumbai", experience: "4-7 yrs", salary: "", type: "", skills: "Big Data", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb17", title: "Specialist UX/UI Designer", applylink: "https://www.techgig.com/jobs/Specialist-UX-UI-Designer/60215", jd: "", companyname: "Hewlett Packard", location: "Bengaluru/Bangalore", experience: "5-9 yrs", salary: "", type: "", skills: "UI / UX", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }, { _id: "5b377db0c97f730014b6eb18", title: "Hadoop - Big Data Developer", applylink: "https://www.techgig.com/jobs/Hadoop-Big-Data-Developer/60225", jd: "", companyname: "Morgan Stanley", location: "Mumbai", experience: "4-7 yrs", salary: "", type: "", skills: "Big Data", startdate: "", enddate: "", created: "", source: "techgig", timestamp: 1530363306.1030896, __v: 0 }];


let {skillsList, locationList, experienceList} = data.reduce((acc, {skills, location, experience}) => {
    acc.skillsList.add(skills);
    acc.locationList.add(location);
    acc.experienceList.add(experience);
    return acc;
}, {skillsList:new Set, locationList:new Set, experienceList:new Set});
skillsList = [...skillsList].map(value => ({value, label:value}));
locationList = [...locationList].map(value => ({value, label:value}));
experienceList = [...experienceList].map(value => ({value, label:value}));
console.log(skillsList);
console.log(locationList);
console.log(experienceList);

Sign up to request clarification or add additional context in comments.

3 Comments

So basically, what you are doing here, is getting all 3 by reduce, making 3 new set,and then putting all of it in there, right ?
Ok cool, really thanks for the help, let me see, if I can optimize it further, with everything in a single map, I have an idea, if I am able to implement, will post the solution
Nope, wasn't able to, thanks for the solution, you are awesome

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.