Acquiring the Data

Qualifications and requirements for usage

  • The Panel Data Research Center, Institute for Economic Studies, Keio University (PDRC) provides microdata for nonprofit and academic purposes only. These microdata are provided to researchers, undergraduate and graduate students (including research students, researchers, etc.)*1 affiliated with a university or research institution*2.

    *1 Undergraduate students cannot have access to the regional data of the Japanese Panel Survey of Consumers (JPSC).
    *2 In principle, data cannot be provided if a researcher affiliated with a for-profit organization requests its use as part of the organization's business activities.

  • It is mandatory to abide by the rules in the pledge when using the PDRC’s microdata. Please give this pledge your utmost consideration and adhere to its requirements. Violation of any pledged items may lead to prohibitions on data use in the future.

Data release policy

1.Release schedule

In principle, the Japan Household Panel Survey (JHPS/KHPS) data are made available for general research purposes approximately one year after the survey period.

2.Data protection and confidentiality policy (JHPS/KHPS and GEES)

Aiming protect the survey respondents, the PDRC adopts a data protection and confidentiality policy. The geographical codes for prefectures and municipalities were deleted and only eight regional codes (Hokkaido, Tohoku, Kanto, Chubu, Kinki, Chugoku, Shikoku, and Kyushu) are available in the provided data. Additionally, since January 2015, all birthday information have been deleted to strengthen the confidentiality.

3.Data provision and costs

The data are made available to be downloaded by approved users directly from our web-based Data Management System (DMS) (except for JHPS-CNEF data that should be downloaded in the Investigator).
There are no costs for using the data.
Students applying for data have to inform if they are undergraduate or graduate students. Research students, researchers, postdoctoral researchers, etc., are considered as graduate students.
When downloading the data on the DMS, the user will receive a folder that contains the microdata files, the survey questionnaires (PDF file), the codebooks (Excel format), and summary tables (Excel format). The microdata is provided in a comma separated value format (*.csv).

Manual

For more details on how to apply for the data, please refer to the manuals below.

Procedures for requesting and using our data

1. Requesting data using the Data Management System (DMS)

It is necessary to create an account before applying for a data. On the PDRC homepage, you can click on “Register / Log in for data application” (on the upper right corner of the screen) and fill out the requested information to create an account. After completing the process, you can login on the DMS and create a new project to request the data.

The registered project will be carefully examined and the PDRC will decide if it will provide or not permission to use the required data. The applicant will receive an examination result notification on the registered email address. It may take several days from the application of the project until the examination result notification. Please allow enough time when planning your research schedule.

* Data release notification and information about conferences and seminars organized by the PDRC may be sent to the registered email address. If you do not want to receive this information, on the DMS profile please choose “Change Profile” and on “Notification email” choose “Not Receive”.
* Please note that important e-mails may be sent even if you have selected "Not Receive".

2.Downloading the data

Once the project is approved, the data can be downloaded from the DMS. Please login on the DMS and download the requested data. It is possible to download the data more than one time if you do it during the allowed period of use.
* JHPS-CNEF data should be downloaded from the Investigator.

3.Data use

Please use the provided data for nonprofit and academic purposes only.

Please inform the PDRC if you find any mistake in the provided data and materials.

Users are responsible for any data interpretation beyond the information available on the homepage and related materials contained in the downloaded files.

Please be sure to clearly state that you received the data from the PDRC when presenting or publishing your research results in academic journals, academic societies, media articles, etc.

Example: The data for this analysis, Japan Household Panel Survey (JHPS/KHPS), was provided by the Panel Data Research Center, Institute for Economic Studies, Keio University.

Use of Generative AI and Related Technologies

The individual-level data provided by the Center, as well as data at the survey-unit level that have been processed or extracted from such individual-level data, must not be entered into or uploaded to generative AI systems such as ChatGPT or large language models (LLMs) for analysis or processing. This restriction also applies to services contracted by universities or research institutions and to services that do not use submitted content for AI training. For the time being, the use of individual-level data with locally installed or on-device LLMs that do not transmit data externally is also prohibited. Exceptions will not be considered at this time.

When using software, code editors, or other tools equipped with AI features, please disable those features, restrict their access, or take other appropriate measures to ensure that individual-level data are not automatically read, indexed, processed, or transmitted externally. The same precautions apply to screens displaying data, images, execution logs, and error messages that may contain data.

On the other hand, creating or modifying program code for analysis is permitted, provided that individual-level data and other non-public information are not supplied to generative AI systems. Publicly available questionnaires, codebooks, and similar materials may also be used, subject to applicable copyright and other terms of use. If entering analysis results into a generative AI system, please ensure that the results contain no information about individual survey subjects or small numbers of respondents and that the content is suitable for public release under the Center’s terms of use.

▽ FAQ
Q1. May I use generative AI to write code in Stata, R, or other programming languages?
A1. Yes, provided that the code-generation request does not include individual-level data or other non-public information. Use publicly available variable information or fictional examples that are not derived from actual response values. Users must review the generated code and execute it in an appropriate environment. They must also ensure that the AI does not automatically access actual data or execution logs.
Q2. May I paste a few rows of data into a generative AI system to investigate the cause of an error?
A2. No. This restriction also applies to data consisting of only a few rows and to data from which identifiers have been removed. Please reproduce the problem using fictional examples that contain no actual data values.
Q3. May I use AI services contracted by my university that do not use submitted content for training, or local LLMs running on standalone devices?
A3. No. The use of individual-level data is prohibited in both cases. This restriction applies regardless of whether the service is free or paid, the type of contract, whether submitted content is used for training, or whether the LLM runs locally.
Q4. May I ask a generative AI system to explain regression coefficients or summary tables?
A4. Only information that may be publicly released under the Center’s terms of use may be entered. Do not enter information if survey subjects could be identified or their information inferred from individual-level predicted values or residuals, tables based on a small number of cases, extreme values, or similar information.
Q5. May I use an editor with built-in AI features?
A5. You must prevent individual-level data from being automatically read, indexed, processed, or transmitted. Take appropriate measures, such as disabling AI features, separating storage locations, and restricting access permissions. Also take care to prevent data from being accessed through screens, logs, or other means.
Q6. Are conventional statistical analyses and machine learning also prohibited?
A6. Conventional analyses conducted in compliance with the applicable terms of use are not prohibited across the board. However, individual-level data must not be read or processed by an LLM as part of such analyses.

4. Until the expiration of the permitted period of use

As a rule, the applicant receives a one-year period permission (that starts from the day the application was approved) to use the data. However, students’ permission terminate by the end of the Japanese academic year (31st March).
Please delete all downloaded data by the end of the period of use.

5. Report of research results

You are required to report the research results you achieved using the data provided by the PDRC. Please login on the DMS and submit your results on “Research Result” option. The research results can be reported during the data usage period or even after the usage period expired.
Even if there are no research results that can be submitted or if you did not use the data, please report it.
If you do not report your results, you may not be able to apply for data usage on the next occasion.

Important points when requesting our data and reporting research results

1.Requesting Data

1) Login on the Data Management System (DMS)
On the PDRC homepage, click on “Register / Log in for data application” on the upper right corner of the screen to create a new account or to login on the DMS.

2) Create a new project
After login on the DMS, click on “New Project” on the upper left corner of the screen and fill out the details of your project.
* Please update your profile with the latest information before submitting a project.

3) Project Information

  • Research Name (Research Theme)
    Choose a suitable title that describes your project.
  • Questions of interest
    After checking the questionnaire, please indicate the relevant questions for your study.
  • Purpose of your research
    Provide the research objectives, the expected results and significance.
  • Research proposal
    Specify the research plan and method for achieving the research goals.
  • Project Referee (for students)
    If the applicant is an undergraduate or graduate student (including research students, researchers, and postdoctoral researchers) it is necessary to indicate a referee. Please, fill out the referee’s information (name, organization, and email address). The referee can be a professor with an academic administration position in your university (department head, dean, rector, etc.), your academic supervisor, or a member from the PDRC. The PDRC will contact the referee by email. Please inform the referee beforehand.
    The referee must create a DMS account in advance.

4) Project Member
In case of joint research, add all the project participants. The project members also need to create a DMS account in advance.
If a project member does not have an account in the DMS, you can use it to send an invitation email.

5) Data Application
Please apply for all data that you will use.
For available data, please check the “List of data sets” homepage.
* Undergraduate students cannot have access to the regional data of the Japanese Panel Survey of Consumers (JPSC).
* Please use the latest available data. If for some specific reason you need an older version of the data, please write the reason in the "Research proposal" space.

6) Application Review Screen
A confirmation screen with the application details will be displayed.

7) Agreement
After reading the pledge, please check the checkbox besides “I accept” and click on “Apply”. By doing so, we will consider that you have accepted the pledge and apply for the data.

Agreement

To,
The Center Director, Professor Isamu Yamamoto
Panel Data Research Center, Institute for Economic Studies, Keio University

PLEDGE

I hereby pledge to observe the following norms while using the data provided by the Panel Data Research Center, Institute for Economic Studies, Keio University (hereafter, “PDRC”).

  1. The Data provided by the PDRC, including the deposited Data, will only be used for the research purposes specified by the Applicant in the Application form and will only be used for academic purposes.
  2. In using the data, every effort will be made to protect the privacy of respondents. I will not engage in any activities that will reveal the identity of individual respondents and will not present or publish any results that make it possible to identify individual respondents. Furthermore, I will not present or publish any results that make it possible to identify the prefecture or region of respondents.
  3. The Data will only be used by individuals who have signed the PDRC’s “Data Request form” as an Applicant or as a member of a project.
  4. The Data will not be provided to any third parties.
  5. The Data and any individual-, household-, or other survey-unit-level data created from the Data will not be input into, transmitted to, or uploaded to any service or software that uses generative AI, large language models (LLMs), or other similar functionality, nor will such data be processed by such service or software. This restriction applies regardless of the contractual arrangement, whether or not the data are used for training, and whether the AI functionality is provided through an external service, hosted within the institution, or run locally on a device. Necessary measures will also be taken to ensure that such AI functionality does not automatically access, transmit, or process the Data.
  6. The Data including any copies made of the Data will be erased by the end of the approved use period.
  7. If the Applicant changes his/her position or affiliation before the end of the approved use period, he/she must promptly report the change to the PDRC using the Data Management System (hereafter, “DMS”).
  8. The Applicant must state that the Data was provided by the Panel Data Research Center (and also its bailor if the Data is deposited to the PDRC) when publishing or presenting papers or any other type of publications based on analyses of the Data. Besides this the user must report to the PDRC submitting a file of the publication using DMS. In case of presentations, the Applicant must use the DMS to submit the contents of each academic conference presentation.
  9. The authors (including co-authors) of papers, which are based on analyses of the Data, will be limited to the Applicant and Joint Users, and such papers will not have any other authors.
  10. The PDRC, along with the bailors of the Data, will not be held responsible for any damages that may arise from the use of the Data.

I have no objections if Keio University institutes legal proceedings in cases when the above pledges are not observed.

NOTES * Refer to the following sample acknowledgement of the Data provision.

Example 1: “The data used for this analysis, Japan Household Panel Survey (JHPS/KHPS), was provided by the Panel Data Research Center, Institute for Economic Studies, Keio University.”

Example 2: “The data used for this analysis, Panel Data of Business Start-up, was deposited by the Japan Finance Corporation and was provided by the Panel Data Research Center, Institute for Economic Studies, Keio University.”

NOTES * Papers and other publications that are based on analyses of deposited data will be reported to the bailors by the PDRC.

2.Reporting research publications

1) Login on the DMS
On the PDRC homepage, click on “Register / Log in for data application” on the upper right corner of the screen to create a new account or to login on the DMS.

2) Choose a project
After login on the DMS, select from the “Project List” the project for which you would like to report a publication, and then click on “Research Result”.

3) Research publication report

  • Status of your research
    Identify if your research is already published or still in progress. If your research was published, please upload a copy of it (PDF file).
  • Research publication information
    Fill out the necessary information about your research publication (authors, title, type of publication, title of the media that published your research, URL of the publication, publication date, and abstract). If the PDF file already has an abstract, this field can be omitted.
    If you have more than one research publication, you can add extra publications by clicking again on “Add Research Publication Report”.
    Even if you do not have research results that can be submitted or if you did not use the data, please indicate it in the “Abstract” field.
    It is possible to report research results even after the permitted period of use expires. If you do not report research results, you may not receive approval to use the PDRC’s data in the future.
  • Use of the research publication report
    Please note that the information provided in your research publication report (authors, publication title, title of the media that published your research, etc.) will be made available on our website (the full text of the publication will not be displayed).

The Data Management System (DMS) was financially supported by Japan Society for the Promotion of Science, Program for Constructing Data Infrastructure for the Humanities and Social Sciences.

About the registered user configuration

We may send data release announcement, information on conferences and seminars hosted by the PDRC and other notifications to the registered email address. In case you would not want to receive these emails, on the DMS profile please choose “Change Profile” and on “Notification email” choose “Not Receive”. (Please be aware that even choosing “Not Receive” we may send important emails).

On the DMS “profile change” homepage is possible to delete your registration. However, please be aware that you cannot delete your registration if there are incomplete projects. Incomplete projects are projects that were not ended, projects that did not expire, or projects whose research results have not been registered.

Questions?

If you have questions, please contact us using the General Inquiries form.
When inquiring about a project that was already created or applied in DMS, please enter the project ID.

Former page

The former page of “Acquiring the Data” (until 2023/03/31) is available here.