Below are a set of frequently asked questions (FAQ). Please feel free to leave comments at the end of this page (we will address your comments as soon as possible).
What's the biggest advantage of using the 10-10 Program vs Performance Assessment System (PAS)?
The main advantage of 10-10 is that it will provide performance assessment information during project execution. This will allow you to identify problems and take corrective action to improve ongoing projects. PAS, on the other hand, provides feedback at closeout, which allows improvement in future projects. Considering that, companies might decide to use both approaches in order to benefit ongoing projects, as well as future projects.
Who can access the 10-10 system?
The 10-10 system is available free of charge to all CII member organizations. In principle, anyone who has a CII account has access to the system. However, the system also requires users to be assigned to a CII Benchmarking role. There are three Benchmarking roles: Benchmarking Manager, Benchmarking Associate, and Benchmarking Project Manager. If you have attended a CII Benchmarking training, you are automatically assigned a Benchmarking Associate role and should be able to log in to the system. If you have a CII account but are unable to log in to the 10-10 system or if you are not sure whether or not you have a benchmarking role, please send an email to UserSupport@cii.utexas.edu .
How would you integrate the 10-10 Program with the CII Project Health Tool?
The 10-10 Program can be seen as an update of the PHI tool, one that allows comparing projects to other projects rather than only to a preset standard. The researchers behind PHI remarked that there is a lack of literature/prior research on leading indicators. PHI and this Program are therefore predominantly based on past CII research and surveys of CII members.
Creating and setting up projects
How can I create a new project and who can create it?
You will find instruction on how to create projects at the 3. Creating a Project page.
Who can see my projects?
Anyone with your company who has a CII account and a benchmarking role will be able to log in to the 10-10 system and see all projects submitted by your company. Note that the team members who only respond to the project survey and who do not have a CII account and benchmarking role will not be able to see the projects and projects reports.
Can I submit a single project phase? Is there value in submitting a single or only a couple of phases?
Yes, you can and we believe that there is value in submitting one or two phases. You can use the 10-10 process to promote communications and identify problems throughout the phase by applying the questionnaire at multiple points.
What does a ‘Project Coordinator' do and who can be assigned as a project coordinator?
The Project Coordinator (PC) is the person who effectively enters the project data (more specifically the data for the general information and output sections). He/she also enters the project team contact information and manages the responses to the input section surveys. Anyone with a CII account and a benchmarking role can be set as a Project Coordinator. Note that each phase of the same project can have a different PC and that the PC can be changed at any time.
What is the range of Total Project Cost (TPC) you are looking for and what will the project size segregation be for reporting?
There is not a project size limit and, differently from the CII General Program, 10-10 does not use project size as a parameter for reporting.
How long will it take to fill out a survey?
We estimate that, for a Project Coordinator responding to all three sections, it should take approximately 2 hours, assuming you already have all the required information including project capacity, cost and schedule data. For a project team member responding only to the input section survey, their input should take 30-40 minutes and can be taken in most phones, tablets, and desktop systems (including IE 9 or later).
What is the best timing to distribute the input section survey for a project that has overlapping phases?
Regardless of overall project timing, each 10-10 survey is designed to be administered and submitted at, or near, the completion of the specific phase. You can open the phase submission and distribute the survey at several points, which will provide you with partial snapshots of the project performance, but the final submission should reflect the actual phase outcomes and should be submitted after the actual end date of the phase.
Where can I find the definitions of phase start and end?
You can find the definitions in the 10-10 Glossary under the entry 'Phases'.
The general information section ('GEN')
Who should respond to the general information section?
The Project Coordinator is the person responsible for entering the general information data.
I can’t find a project type that matches my project. What should I do?
If you don't think your project fits one of the default project types available in the 'GEN' section, you can always enter a custom project type but, in that case, the system will not be able to generate several output measures that depend on having a minimum number of projects sharing the same project type. (Your project might be the only one with the custom project type). You can contact us and consult about what project type is the most adequate for your project.
My project is an international project, but there is no appropriate currency in the “Currency” dropdown list. What should I do?
You can skip the 'Currency' question and contact us requesting your project’s currency to be added to the default list. In order to leave the 'GEN' survey without losing your data, you must click on the 'Save & Exit' button before you leave the page so you can resume the data entry later on. Note that you can still respond to the 'Exchange rate' question (two questions after the 'Currency' question) even though your currency is not listed in the 'Currency' question.
Can I choose a specific date for the midpoint of phase schedule that is not the actual midpoint of the phase dates reported in the output section?
Yes you can. After you submit the project to CII, your Account Manager (responsible for validating the data you submitted) will probably ask you if that is indeed correct (to make sure it was not a typo) and you can just confirm that the date is correct.
The input section
Who are the team members that should be included in the input section survey?
Should I include team members that work for contractors if I work for an owner organization?
The team members should be all the people that are part of your organization and are involved in the phase. For an owner organization, the team members should include only people working directly for the owner organization (therefore excluding people working for contractors and consultants). For contractors, the team members can include both the people who work directly for the contractor, as well as subcontractors that are managed by your organization.
Should the survey team remain the same throughout the several phases of a project?
If not, would this change the validity/evidence of how well we’re doing or the outcome of the results?
The team members do not have to remain the same across the project phases. Our primary concern is to have a representative sample of the people working in each phase. At the moment, we are not able to assess if there are differences between the two cases because we have very few cases with the same people responding every phase (except the many cases where only one person responded to the survey). In fact, each phase team is expected to be different since different people are expected to be involved in each phase (e.g., the FEP and construction teams might have some people in common, but are expected to be different teams).
Is the survey anonymous?
Yes, the responses are aggregated by question in the report and cannot be traced back to the individual respondents.
Can I aggregate the team responses and submit a single survey?
CII encourages project team members to submit their surveys separately and anonymously and the system was developed to receive separate responses. This is mainly intended to avoid biases that might emerge when responses are discussed in a group setting.
After a survey round has been started, how long should I wait for the responses?
The default time window for team members is two weeks and, as of January 2015, it cannot be changed by the users. Note that the Project Coordinator can restart the round at any time but any partial survey data collected up to that point will be lost.
Can the initial survey be re-opened for the other team members to complete?
I closed a survey round, but now I want additional input from the team.
No you cannot re-open a survey in order to ask more people to respond. However, during the two-week period of an ongoing survey round, you can add new emails and the system will send out automatic requests to the added team members.
You can also start a new survey round, which might include new team members. However, when you start a new survey round, the existing data will be discarded and all team members will be asked to respond to the survey again. We recommend you to save the current results as a pdf [create entry on how to showing how to create it] and wait until the right time to open and start a new round.
Can I have multiple rounds for the input section survey?
Yes but note that with each new round, the previous data will be discarded. You should save the partial report with the current round data as a pdf before proceeding to open a new round.
When creating teams for the input section survey, is there an 'address book' feature to help me expedite the entry of team members?
No, there is not, but this is a feature that we consider to add in the future.
Are there any definitions or clarifications to answering the statement-based questions?
We have released a 10-10 glossary, which is also available as a link in the online questionnaire. It’s intended as a companion for the website since we have not implemented an online help capability. However, you can always contact us if you have question and we will keep extending the documentation in order to address the issues that we identify as ‘frequently asked’.
Do the questions change for each phase or are they consistent across the phases?
There is a common set of questions across the phases but questions do change slightly for each sector/phase.
What should I do if I am not sure about a specific question in the input section survey or if a question does not apply to my scope of work?
Team members responding to the input section survey can skip questions whenever they think questions do not relate to their scope of work or when they are not sure about the response.
How do we get feedback after a survey round?
After the Project Coordinator closes a survey round, a 'preliminary report' becomes available. This report will show how the team members responded, as a group, to each input section survey's questions. The 'preliminary report' will not contain any input or output measure charts since these only become available after the project is validated by CII. The preliminary report can be printed or saved as a pdf file. Note that, when a new round is started, the previous data is discarded. Therefore, if you want to retain multiple versions of preliminary reports (in case you decided to take multiple rounds of the input measure survey), you should save each version as a pdf or print a hardcopy of each version.
The output section
I do not have the phase cost data (or capacity or schedule). What should I do?
There are no mandatory questions in the survey so you can always skip questions. However, most of the questions in the output section are required for calculating the output measures. The output measures that depend on the missing data point will not be calculated. If you think you can obtain the missing data later, you can always save the current data and return later to add the missing data. If you want to resume the data entry later, click on the 'Save & Exit' button in order to save the data you already entered. Please do not click on the 'Complete' button on the last page of the 'OUT' section because that will close the data entry for that section.
Where can I find the definition of costs included in the phase cost?
We provide a list of items commonly included in each phase cost in the 10-10 Glossary, under the 'Phases' entry. See the 10-10 Glossary.
From an owner's perspective, will the engineering man-hours include both external consulting engineers and its own engineering team?
For owners, the reported man-hours should include only the owner’s personnel hours. For contractors, the hours should include any work of engineering managed by the organization, which might include subcontractors and consultants' hours.
I can’t find a capacity unit that fits my project units. What should I do?
If you don't think your project's capacity unit fits one of the default units available in the 'OUT' section, you can always enter a custom project unit but, in that case, the system will not be able to generate several output measures that depend on having a minimum number of projects sharing the same capacity unit. (This is true of all 'efficiency' measures that depend on capacity unit). You can contact us and consult about what capacity unit is the most adequate capacity unit for your project.
I am not able to submit the project to CII. What should I do?
One possible reason is that you need to complete all three sections ('GEN', 'IN', and 'OUT') before submitting a project to CII. Please review your data and make sure you clicked on 'Complete' at the end of the 'GEN' and 'OUT' sections and that you closed a survey round in the 'IN' section. If after doing so, you are still unable to submit you project, please contact us for assistance.
Visualizing, understanding or using results
When can I see my report?
The final 10-10 report becomes available after the project is submitted to and validated by CII staff, and the project database is refreshed (every Wednesday night). Before that, only a 'preliminary report' is available, which shows the team responses to input section questions but does not provide any quartile charts.
Are the results shown by industry, e.g. natural gas, mining, oil etc. ?
Yes, the 10-10 output charts will be generated for different project types. Also, projects can be grouped by project nature (greenfield, modernization, etc.) as has been used in CII’s General Program.
Are there any guidelines to understand which questions in the survey are associated with in the 10 input/output scores?
I scored poorly in an input measure, say Planning. How can I figure out what’s wrong?
Currently, you can use the 'Questions-Input Map' file to identify which questions are related to each input measure (use the following link to download it: CII 10-10_Question_Map_Jul_2015_R0.pdf). There is one Question-Input Map for each sector/phase since the set of questions changes for every sector/phase questionnaire.
How are the input measure scores calculated? How are the output measure scores calculated?
The input measures are calculated based on the team member responses to the questions associated with each input measure (use the following link to download the question-to-measure map: CII 10-10_Question_Map_Jul_2015_R0.pdf). Each question received a weight associated with each input measure, which reflects how strongly the question is related to the measure (note that these weights are not included in the file above). The measure score is a result of the sum of the product of each question's score (which depends on the team member responses) and the question weight to that specific measure (which is pre-defined in the system).
The output measures are calculated based on the measure formula. See 10-10 Measures - Definitions.
After receiving a report, is there a way to identify which CII tools will help me to address the problems I identified?
Currently, there is not. However, this is a future development planned for the 10-10 Program.
How can I interpret an Input/Output Measure quartile chart?
An input measure quartile chart will show you how your score compares to the set of similar projects in the database. Your score is indicated by the black dot in the chart (with the score written in the white call-out box). That score indicates how much the team answers scored out of the possible total of 100%. For instance, in the figure below, the planning score indicates that the answers to the questions related to the planning measure scored 53% of the possible 100%. That score can be compared to the score of similar projects by observing the fours quartiles in the chart. In the figure below, the project planning score puts it in the fourth quartile, among the 25% of the projects with the worst planning scores, compared to the set of 12 similar projects found in the database.
For output measures, the black dot indicates the score calculated according to the measure formula (see 10-10 Measures - Definitions for the formula associated with each output measure). The four quartiles will indicate how that score compares to the score of similar projects in the database. In the figure below, the score of 9% puts the project cost growth very close to the median of the 10 similar projects (roughly 50% of the projects have worse scores and 50 % have better scores).
What does the “C” indicate in my report?
I can’t see any quartiles. What’s wrong?
The 'C' symbol indicates that the system was not able to generate a quartile chart.
There are two reasons why a quartile chart might not be present:
i) The 10-10 system was not able to find a minimum of 8 similar projects submitted by at least 3 companies. This is particularly important for measures based on capacity, which require similar projects to share the same capacity unit, a requirement that rapidly reduces the number of similar projects. This is also an issue for the infrastructure sector, which has a smaller number of submitted projects compared to the industrial and building sectors.
ii) The data points required for calculating the measure are missing. Please check your output section data and contact your account manager if there are missing data points that you want to provide.
What projects are included in my quartile charts?
What does the “All” mean in the comparison criteria section?
The 10-10 system automatically searches for the set of similar projects to be included in each quartile chart. To understand how these projects were selected, the user needs to read the 'Input Measure Comparison Criteria' and 'Output Measure Comparison Criteria' tables provided in the report (see figure below). The 'Input Measures Comparison Criteria' table below shows that the selected projects are all Industrial projects in the construction phase, with the same project type ('Electrical') and include both owner and contractor respondents (the respondent field is 'All', which indicates that all respondent categories are included in the chart).
How can I print the report?
You need to use your browser's printing capability in order to print the reports. For most (Windows) systems, you should be able to right click anywhere on the report and choose 'print'. If you have a pdf printer installed (normally, if you have Adobe Acrobat installed, you will automatically have a pdf printer installed), you can choose the 'print as pdf' option, which will allow you to save your report as a pdf file.
Will there be data mining capabilities for 10-10 like in PAS?
A Data Miner for 10-10 is not part of this first phase of the project. However, it is CII’s intent to include it in the second phase of the 10-10 Program and have it available by the end of 2015.
Will the 10-10 data be available for each geographic region?
Yes it will. Some regions might have few projects initially, but we expect most regions (continents, countries, and US States) to be well represented in the 10-10 database in the long run. By the end of 2015, regions will be ‘filter’ criteria in the 10-10 Data Miner as well.