User Survey Summary
Additonal Information
- EMSL Survey Results [.jpg, 195kb]
- Previous years survey results
EMSL is committed to continually improving the users' experience. Although change cannot always be implemented overnight, we rely strongly on the input received from our user community and encourage you to continue providing feedback to our technical staff, our User Support Office, and through the bi-annual survey. Users can also provide comments and feedback to the User Executive Committee and should feel free to contact anyone on the committee at any time.
Currently, user surveys are administered biannually for experimental users and are sent only to those individuals who have accessed our resources during the prior six months or annually for computational-only users and are sent only to those individuals who have accessed computing resources during the prior year. The results of the most recent survey are posted here with management responses to concerns or issues identified by our user community.
October 2011 Survey
Surveys Submitted Between October 11, 2011, and October 31, 2011.
Survey Satisfaction: 91.5 %
Survey Responses: 208
Surveys Sent: 619
Survey Response Rate: 33.6%
1. How satisfied were you with the availability of facilities and equipment?
- 116 Very Satisfied
- 80 Satisfied
- 8 Neither Satisfied nor Dissatisfied
- 2 Dissatisfied
- 1 Very Dissatisfied
- 1 Not Applicable
2. How satisfied were you with performance of facilities and equipment (e.g., were they maintained to specifications for your intended use, ready when scheduled, etc.)?
- 126 Very Satisfied
- 73 Satisfied
- 5 Neither Satisfied nor Dissatisfied
- 2 Dissatisfied
- 0 Very Dissatisfied
- 1 Not Applicable
3. List additional capabilities that you think EMSL should have.
User comments to this and other survey questions are below.
4. With the new knowledge gained at EMSL, I expect to (check all that apply):
- 181 Disseminate new knowledge via publication in peer-reviewed open literature
- 148 Disseminate new knowledge via presentations at professional society meetings
- 11 Acquire a patent
- 69 Further Department of Energy mission(s)
- 116 Facilitate collaborative interactions (e.g., stimulated new ideas for future experiment; increased work; etc.)
- 67 Train students (undergraduate, graduate or postdoctoral associate)
- 114 Use data for a future proposal
- 80 Establish or grow network and/or further collaboration
- 3 Other
5. How satisfied were you with the assistance provided by the EMSL technical staff?
- 143 Very Satisfied
- 52 Satisfied
- 6 Neither Satisfied nor Dissatisfied
- 2 Dissatisfied
- 0 Very Dissatisfied
- 2 Not Applicable
6. How satisfied were you with the assistance provided by the EMSL administrative staff?
- 130 Very Satisfied
- 48 Satisfied
- 12 Neither Satisfied nor Dissatisfied
- 1 Dissatisfied
- 0 Very Dissatisfied
- 16 Not Applicable
7. How appropriate and user friendly were the training and safety procedures?
- 57 Very Satisfied
- 82 Satisfied
- 26 Neither Satisfied nor Dissatisfied
- 7 Dissatisfied
- 2 Very Dissatisfied
- 33 Not Applicable
8. How satisfied were you with the proposal process (e.g. submission & review)?
- 59 Very Satisfied
- 73 Satisfied
- 16 Neither Satisfied nor Dissatisfied
- 3 Dissatisfied
- 3 Very Dissatisfied
- 51 Not Applicable
9. How did you learn about EMSL?
- 26 Scientific meeting/conference
- 6 Internet search
- 10 Journal publication
- 59 Previous EMSL use
- 85 Colleague
- 101 PNNL staff member
- 9 Other
USER ENDORSEMENTS AND COMMENTS
Although not all comments are shown, below is a representative sampling of the positive user comments received.
- I appreciate that the operators of the instruments that I needed to use made time for me as well as took the time to answer my questions.
- The availability of 850 and 900 MHz NMR spectrometers makes a great impact on our ability to do science. These are unique spectrometers.
- World class.
- They [the staff] are all awesome and very professional. They all have the knowledge and resources for problem solving and enabling science to move forward.
- I was surprised how easy it was to schedule my experiments on the instruments!
- The data collected for our work was of high quality.
- My arrival was preceded by a cyber attack on PNNL so things were a bit of a mess. However, the secretarial staff I was in contact with at EMSL went out of their way to get me into the system as quickly as they could for the short time I was there and I appreciate that.
- The user support staff are excellent and very helpful!
- The website is easy to use, which makes submitting the proposal as easy as possible.
- The [proposal] process is straightforward with deadlines and application procedures well described
- This is a fantastic resource that should continue to be supported. It is essential as a resource for the USA to remain competitive in a global research environment.
User Concerns and Suggestions
Comments in which users expressed concerns or suggestions for improvement have been compiled into several topical areas. Representative comments are provided below, along with EMSL's response.
User Comments
Oversubscribed Instrumentation or Staffing
- Mass spectrometry waiting time is a bit long. Samples are waiting in the freezer until expired and we still did not get the instrument time to analyze them.
- It takes a long time in queue for the proteomic analysis
- My only criticism is that proteomics samples have such a long turn over time. Upon submission they seemed to be prepared quickly, but then spent months "in the queue". My group is still waiting on samples submitted over eight months ago. I know that there are many users and samples must be prioritized in order of arrival or necessity, and my complaint is not directed at the staff receiving or preparing my samples for the queue.
- I found the walltime limit of 48 hours for non-resource intensive jobs on Chinook sometimes to be limiting.
- Somewhat unhappy about change in policies that resulted in loss of data. The way PNNL staff are treated is particularly poor in these regards - it would appear that we are treated just the same as external users even though, in fact, we report some things internally, e.g. through ERICA.
- Is the instrument and EMSL staff dedicated to metabolomics a worthwhile investment? How many papers after so many years have come of this?
EMSL's Response
EMSL continues to work on minimizing long queues for run times and analysis with new staff hires and instrument acquisitions—especially in the mass spectrometry capability. As we mentioned in the last survey, we're making every effort to reduce queue times in the mass spectrometry capability, and with funding largely from the American Recovery and Reinvestment Act, we added additional measurement platforms to increase throughput. We are also adding additional monitoring and tracking capabilities to our informatics system to enable more optimal distribution of the work load across our resources, and we think you'll see improvement this fiscal year.
We seek to address scientific needs relevant to many disciplines and sub disciplines within our user community, including those areas that require capability development prior to full implementation. The biological user community has communicated their desire for a magnetics resonance metabolomics capability, and we are working towards meeting this need. Currently only one of the five liquid state NMR spectrometers is dedicated to this endeavor, leaving the remaining four spectrometers to meet the needs of the structural biology community. Staff support given toward metabolomics work is directly tied to the user proposals that were favorably reviewed and approved in this area as part of our FY2011 Call for Proposals cycle.
For our users who are also PNNL staff, we understand it can be frustrating to be required to report information to both your organization and to the user facility. In many cases, EMSL has been able to integrate our reporting system with PNNL's data systems. We are currently working with the ERICA system owners to investigate the possibilities and costs of further integrating our systems to allow ERICA to push your latest publication entries to your EMSL proposals. Our hope is to implement this improvement this fiscal year.
In the NMR capability, we now conduct basic testing of the equipment requested prior to the user's arrival. Due to the complexity of the spectrometers and the breadth of experimental setups requested by users, it is also sometimes difficult to switch between operational modes within a short time frame. To address this, we have increased the time allocated for switching the spectrometer between one user and another, and we anticipate a significant decrease in downtime experienced by the user. In addition, to address the drift on the 900 MHz NMR, a solid-state lock capability has been developed and deployed for the bio-MAS probe. A second solid-state lock is being developed for the 1.6 mm HXY probe, also on the 900 MHz.
User Comments
Training
- Repeating the same computer security training every year did not add much to what we did.
- While I understand the frequency at which you must renew your training, it is sometimes tedious to constantly repeat the same modules. This is especially the case for the computer training due to the length of the module.
- You must be sick of hearing this but your training modules are an overkill, at least for the US university users.
- Some of the training was relevant and informative, but a lot was focused on Hanford 300 Area operations etc. that were not applicable. Some redundancy in training modules extends the amount of time it takes to complete the training.
EMSL's Response
We're not sick of hearing this–because we completely understand! The bulk of frustration with user training is related to the extensive and time-consuming PNNL orientation that is required for any non-staff working onsite. In many cases, it is targeted for PNNL employees rather than non-staff. As we reported last year, PNNL began a concerted effort to re-evaluate and re-scope the training plans. As of today, PNNL has identified the base requirements needed for non-staff to work onsite and are in the process of re-writing and re-designing the orientation module. They expect the revised orientation to roll out in mid-April and take less than one hour to complete (a significant reduction from the previous 3-4 hour time commitment).
In addition, we continue to work with Cyber Security on the development of a streamlined course for non-staff, although the final design will wait until after the orientation module is complete. We're also evaluating the courses being assigned to users working in specific laboratories, and have identified some modules that we believe can be streamlined and targeted to only a subset of users and still maintain a safe working environment.
User Comments
Transparent Processes and Communication
- Our proposal was supported by an internal DOE contract, however the proposal system asks for a grant number, grant managers which we didn't know exist for a DOE contract. We indicated that the proposal was part of an already approved DOE contract, but our proposal still went to outside reviewers for the scientific merit which delayed the overall progress significantly. We encourage EMSL to update the proposal system to accommodate projects that are funded by internal DOE contracts.
- Sometimes due to number of proposal, it takes time to get approval on the proposal. Other than that there is no limitation.
EMSL's Response
"EMSL" can be confusing for PNNL staff because we're 1) a PNNL directorate, 2) a building, and 3) a national scientific user facility. While PNNL staff normally work across directorates and buildings without additional requirements, EMSL must follow guidelines established by the Office of Science for national user facilities. These guidelines require a peer-reviewed proposal process, and all proposals (whether from PNNL staff or external non-staff) compete for access based not only on the science but on the fit to BER's and EMSL's mission and science directions. Work package numbers are requested for PNNL staff simply to ensure the proposal type selected meets the project scope (for example, a non-proprietary proposal should not be selected on a restricted project).
Last year, we implemented a new notification/response process for our users to expedite decisions to reassign the proposal to new reviewers if a reviewer cannot complete the review in two weeks. Following a Call for Proposals, we find our reviewers often ask for a break, which can impact the turnaround of a proposal review. Nevertheless, we find that the overall response time is improving. In addition, we've added some internal management tracking activities and will continue to identify ways to expedite the review process.
User Comments
Operational Delays or Downtime
- My work involves radioactive samples and some of the resources were not available because of the nature of my samples.
- The NMR instruments are excellent resources, but access and availability are severely compromised by the continuing DOE/EMSL/PNNL approach to cyber security.
- The temperature control equipment had a poor connection that imperiled our samples with temperature swings.
- We had an issue with [NMR] equipment on the first day that cost one day of data acquisition, but things were fine after that.
- The 900 magnet drift is a severe impediment to collecting high resolution BioSolids data.
- The [NMR] instrument was not ready and it took several hours to get it running in the mode I needed.
- More established routines concerning requirements to send material [are needed]. We spent a lot of time talking to the USDA before we finally [learned] we did not need an extra permit for the protein extracts of a bacterial pathogen.
- I always had difficulty getting on, submitting jobs, and viewing output since the connection over the firewall is terribly slow.
- The acquisition process was fantastic, however, we did not leave with our data and have been unable to obtain it.
EMSL's Response
The Radiochemistry Annex, which would allow users with radioactive samples to conduct their research, is currently being constructed. A number of capabilities have come on-line within the last 6 months, including a dual beam FIB/SEM system and a TEM with an electron energy loss spectrometer. Additional capabilities, including NMR, EPR and XPS spectrometers; electron probe and scanning probe microscopes; and an x-ray diffractometer will become available once construction is complete and these capabilities are installed. We anticipate full operation of this capability in February 2013.
Walltime limits on Chinook have been set to maximize throughput for the majority of users. For jobs with less than 16 nodes, the walltime limit is 96 hours. When there is clearly a need for more nodes and longer run times, the Molecular Computing Capability staff will work with researchers to provide more nodes and/or longer run times. In addition, connection speeds between EMSL computers and those researcher's computers outside our firewall are dependent on many factors--not just our firewall. Depending on the routing, number of segments, server speeds and loads, and the encryption method used all have an effect. We have experts that can investigate the source of slow data transfers and sometimes orders of magnitude improvements have been realized. If you have specific issues or needs related to these issues, please contact Erich Vorpagel at 509-371-6448.
Users who are experiencing problems in obtaining their data can contact the USO with specifics. We'll work with the staff to identify the issues and provide you with an estimated delivery date.
User Comments
Additional Capabilities Requested by the Users
In addition to requests for new or upgraded instruments, users responded with suggestions for additional lab space, improved internet access, and additional software modifications.
- Low Energy Ion Scattering (LEIS)
- A Cryogenic probe on the 900 MHz NMR
- Tools to debug codes like Totalview, ddt, etc.
- Electroplating
- Improved silicon wafer dicing
- Voltammetry for metal speciation study
- Dedicated microscope assisted laser fluorescence and Raman spectroscopy
- Dynamic screen calorimeter for protein screening
- Atomic layer deposition system
- Gaussian or Dalton
- In-situ electrochemical cell for XPS and RBS measurements
- Fractionation collectors
- Low temperature probes for all NMR instruments
- More flexible and better controlled goniometer in tandem accelerator
- PPMS, FMR
EMSL's Response
The Physical Property Measurement System (PPMS) and the Vibrating Sample Magnetometer (VSM) will make measurements similar to those from a SQUID. If you don't see something on the website that you're interested in, or want to pursue specifics related to the experiments you have in mind, we encourage you to ask the Capability Lead (http://www.emsl.pnl.gov/contacts/) or your host.

EMSL's Response
EMSL continues to expand its reputation as a world class facility by coupling unique sets of capabilities with outstanding teams across multiple platforms and disciplines. The power and quality of our resources, such as the high field NMRs, and the expertise and dedication of all the scientific staff are invaluable to our user program. We especially want to call out Heather Brewer, Sarah Burton, Alice Dohnalkova, Nancy Isern, Ravi Kukkadapu, Ljiljana Pasa-Tolic, and Si Wu who received special recognition for their knowledge, commitment, and outstanding help to our users.