microsoft · MerlinSmiles · Apr 15, 2016 · Apr 17, 2016 · Apr 17, 2016 · Apr 17, 2016
diff --git a/.gitignore b/.gitignore
@@ -62,4 +62,11 @@ target/
 # Generated data
 *.csv
 *.data
-*.hdf5
+*.dat
+*.hdf5
+
+# Temporary files
+*.md~
+tmp/
+
+docs/examples/data/*
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -22,17 +22,28 @@ Figured out a new way to use qcodes? Found a package that makes your life better
 
 ### Setup
 
-- Clone and register the package for development as described in [README.md#installation]
+- Clone and register the package for development as described in the [README](README.md#installation)
 
 ### Running Tests
 
-The core test runner is in `qcodes/test.py:
+The core test runner is in `qcodes/test.py`:
 ```
 python qcodes/test.py
-# optional extra verbosity
-python qcodes/test.py -v
+# optional extra verbosity and fail fast
+python qcodes/test.py -v -f
 ```
-You should see output that looks something like this:
+
+You can also run single tests with:
+```
+# python -m unittest module
+# python -m unittest module.class
+# python -m unittest module.class.function
+python -m unittest qcodes.tests.test_metadata
+# or
+python -m unittest qcodes.tests.test_metadata.TestMetadatable.test_snapshot
+```
+
+If you run the core test runner, you should see output that looks something like this:
 ```
 .........***** found one MockMock, testing *****
 ............................................Timing resolution:
@@ -122,7 +133,7 @@ Coverage testing is generally meaningless for instrument drivers, as calls to `a
 
 - Write your new feature or fix. Be sure it doesn't break any existing tests, and please write tests that cover your feature as well, or if you are fixing a bug, write a test that would have failed before your fix. Our goal is 100% test coverage, and although we are not there, we should always strive to increase our coverage with each new feature. Please be aware also that 100% test coverage does NOT necessarily mean 100% logic coverage. If (as is often the case in Python) a single line of code can behave differently for different inputs, coverage in itself will not ensure that this is tested.
 
-- The standard test commands are listed above under [Running Tests](#running_tests). More notes on different test runners can be found in [TESTING.md].
+- The standard test commands are listed above under [Running Tests](#running_tests). More notes on different test runners can be found in [TESTING](TESTING.md).
 
 - Core tests live in [qcodes/tests](https://github.com/qdev-dk/Qcodes/tree/master/qcodes/tests) and instrument tests live in the same directories as the instrument drivers.
 

diff --git a/LICENSE.md b/LICENSE.md
@@ -0,0 +1,26 @@
+Copyright (c) 2015, 2016 by Microsoft Corporation and Københavns Universitet.
+
+QCoDeS is currently available only within the Microsoft Station Q collaboration and by specific private arrangement with a beta test group. Please do not share it with anyone who is not part of the beta test program. If you have a coworker who would like access, please contact Alex Johnson (alex.johnson@nbi.ku.dk) to see if they can be added to the program.
+
+Please report all bugs, limitations, or missing functionality as issues on the Qcodes GitHub.
+
+We encourage you to contribute any bug fixes or new functionality you develop, especially device drivers, by creating a GitHub branch and submitting a pull request. Note that any code you contribute may be included in the public open source release of QCoDeS.
+
+We intend to release it as open source software once it is robust and reasonably stable, under the following license terms:
+
+> QCoDeS is available under the [MIT open-source license](https://opensource.org/licenses/MIT):
+> 
+> Permission is hereby granted, free of charge, to any person obtaining a copy of this software
+> and associated documentation files (the "Software"), to deal in the Software without restriction, 
+> including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, 
+> and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, 
+> subject to the following conditions:
+
+> The above copyright notice and this permission notice shall be included in all copies or 
+> substantial portions of the Software.
+
+> SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT 
+> LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 
+> IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+> WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 
+> SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
diff --git a/LICENSE.txt b/LICENSE.txt
diff --git a/README.md b/README.md
@@ -45,13 +45,7 @@ If you registered Qcodes with Python via `setup.py develop`, all you need to do
 
 See the [docs](docs) directory, particularly the notebooks in [docs/examples](docs/examples)
 
-Until we have this prepared as an installable package, you need to make sure Python can find qcodes by adding the repository root directory to `sys.path`:
-```
-import sys
-qcpath = 'your/Qcodes/repository/path'
-if qcpath not in sys.path:
-    sys.path.append(qcpath)
-```
+For frequently asked questions see the [Qcodes FAQ](docs/FAQ.md).
 
 ## Contributing
 
@@ -61,4 +55,4 @@ See the [Roadmap](ROADMAP.md) an overview of where the project intends to go.
 
 ## License
 
-Qcodes is currently a private development of Microsoft's Station Q collaboration, and IS NOT licensed for distribution outside the collaboration. We intend to release it as open source software once it is robust and reasonably stable, under the MIT license or similar. See [License](LICENSE.txt)
+Qcodes is currently a private development of Microsoft's Station Q collaboration, and IS NOT licensed for distribution outside the collaboration except by arrangement. We intend to release it as open source software once it is robust and reasonably stable, under the MIT license. See [License](LICENSE.md).
diff --git a/TESTING.md b/TESTING.md
@@ -1,6 +1,6 @@
 # Notes on test runners compatible with Qcodes
 
-There is now a test script [test.py] in the root directory that uses the standard `unittest` machinery to run all the core tests (does not include instrument drivers). It has been tested on Mac (terminal), and Windows (cmd, git bash, and PowerShell). It includes coverage testing, but will only print a coverage report if tests pass.
+There is now a test script [test.py](qcodes/test.py) in the qcodes directory that uses the standard `unittest` machinery to run all the core tests (does not include instrument drivers). It has been tested on Mac (terminal), and Windows (cmd, git bash, and PowerShell). It includes coverage testing, but will only print a coverage report if tests pass.
 
 The biggest difficulty with testing Qcodes is windows multiprocessing. The spawn method restricts execution in ways that are annoying for regular users (no class/function definitions in the notebook, no closures) but seem to be completely incompatible with some test runners (and/or coverage tracking)
 

diff --git a/docs/FAQ.md b/docs/FAQ.md
@@ -0,0 +1,33 @@
+# QCodes FAQ
+
+This FAQ is intended for users for Qcodes. For development, see Github.
+
+## Installation
+
+### How to install Qcodes?
+
+...
+## Usage
+
+### What are the different ways to run Loops, and when would you use them?
+
+`Loop.run()` (or `ActiveLoop.run()`) has a two arguments that control where the loop runs and how the data is handled: `background` (`True` (default) or `False`) and `data_manager` (`None` (gets the default `DataManager`) or `False`).
+
+- Default usage: `Loop.run()`:
+  Involves two extra processes, one `<Measurement>` process that sequences the loop actions and a `<DataServer>` process that the `<Measurement>` feeds data to, then stores it to disk and provides it to other processes that want it. This is the normal way to run loops, because it minimizes the work done in the measurement process, so it runs as fast as possible, and also keeps the main process free for other tasks like live plotting and analysis.
+
+- Foreground with a DataManager: `Loop.run(background=False)`:
+  The measurement loop runs in the process that started it, rather than making a new process, so the starting process blocks until the loop is finished. You might do this to make debugging easier.
+
+- Background with no DataManager: `Loop.run(data_manager=False)`:
+  The measurement loop runs in the background, but does not start (or connect to, if one is started already) a `<DataServer>` process; instead, it holds and stores the data itself. If the main process wants to sync this data during acquisition, it will need to read it from disk. Not sure why you would use this mode, but it's possible.
+
+- Foreground with no DataManager: `Loop.run(background=False, data_manager=False)`:
+  No extra processes are involved; the measurement loop runs in the process that started it, and holds and saves the data itself. If you want to start another measurement loop while one is already running (for example if you have a complex parameter that runs its own measurement loop to determine some derived value), you need to use this mode. That's because only one background measurement is allowed at a time, and only one `DataSet` may be on the `<DataServer>` at a time. But this mode will still save the `DataSet` it makes; in most such complex parameter cases you want an even more stripped-down loop: `Loop.run_temp()` which just calls: `Loop.run(background=False, quiet=True, data_manager=False, location=False)` so it does not save anything, nor does it print the normal messages that `run` prints describing the `DataSet` it makes.
+
+### How to abort a running measurement?
+
+Use `qc.halt_bg()`. To list the active measurements use `qc.active_children()`
+
+
+
diff --git a/docs/examples/Load-and-plot-old-data.ipynb b/docs/examples/Load-and-plot-old-data.ipynb
@@ -1,5 +1,30 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Loading data for analysis: the `load_data` function.\n",
+    "\n",
+    "`DataSet` objects are not intended to be instantiated directly, but\n",
+    "rather through the helper functions:\n",
+    "- `load_data` for existing data sets, including the data currently\n",
+    "  being acquired.\n",
+    "- `new_data` to make an empty data set to be populated with new\n",
+    "  measurements or simulation data. `new_data` is called internally by\n",
+    "  `Loop.run()` so is also generally not needed directly.\n",
+    "\n",
+    "If you omit `location`, or if `location` matches the data set currently\n",
+    "being acquired, `load_data` and subsequent calls to `data_set.sync()`\n",
+    "will pull from the `DataServer` (`DataMode.PULL_FROM_SERVER`).\n",
+    "Otherwise `load_data` and `data_set.sync()` will read from disk\n",
+    "(`DataMode.LOCAL`).\n",
+    "\n",
+    "Note that a `DataServer` is, at least for now, local to one parent\n",
+    "process / notebook, so if you open a separate notebook for analysis, even\n",
+    "your live data will be pulled from disk."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -234,7 +259,7 @@
        "    min-height: 50px;\n",
        "    max-height: 400px;\n",
        "    min-width: 400px;\n",
-       "    max-width: 700px;\n",
+       "    max-width: 1000px;\n",
        "}</style>"
       ],
       "text/plain": [
@@ -248,8 +273,8 @@
      "data": {
       "text/plain": [
        "DataSet: DataMode.LOCAL, location='testsweep'\n",
-       "   amplitude: None\n",
-       "   chan0: None"
+       "   chan0: None\n",
+       "   amplitude: None"
       ]
      },
      "execution_count": 1,
@@ -260,7 +285,7 @@
    "source": [
     "%matplotlib nbagg\n",
     "import qcodes as qc\n",
-    "data = qc.DataSet('testsweep')\n",
+    "data = qc.load_data('testsweep')\n",
     "data"
    ]
   },
@@ -1050,7 +1075,7 @@
     {
      "data": {
       "text/plain": [
-       "<qcodes.plots.matplotlib.MatPlot at 0x104005898>"
+       "<qcodes.plots.matplotlib.MatPlot at 0x10410e240>"
       ]
      },
      "execution_count": 2,
@@ -1062,6 +1087,32 @@
     "qc.MatPlot(data.amplitude)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "ename": "RuntimeError",
+     "evalue": "DataManager has no live data",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
+      "\u001b[0;32m<ipython-input-8-22984df78935>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mlive_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mload_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# this will be an error because there is no live data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mlive_data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;32m/Users/alex/qdev/Qcodes/qcodes/data/data_set.py\u001b[0m in \u001b[0;36mload_data\u001b[0;34m(location, data_manager, formatter, io)\u001b[0m\n\u001b[1;32m     85\u001b[0m                                'not exist or was requested not to be used')\n\u001b[1;32m     86\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 87\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_get_live_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata_manager\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     88\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     89\u001b[0m     elif (data_manager and\n",
+      "\u001b[0;32m/Users/alex/qdev/Qcodes/qcodes/data/data_set.py\u001b[0m in \u001b[0;36m_get_live_data\u001b[0;34m(data_manager)\u001b[0m\n\u001b[1;32m     99\u001b[0m     \u001b[0mlive_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata_manager\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mask\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'get_data'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    100\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mlive_data\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlive_data\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mNoData\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 101\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'DataManager has no live data'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    102\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    103\u001b[0m     \u001b[0mlive_data\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmode\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mDataMode\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mPULL_FROM_SERVER\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;31mRuntimeError\u001b[0m: DataManager has no live data"
+     ]
+    }
+   ],
+   "source": [
+    "live_data = qc.load_data()  # this will be an error because there is no live data\n",
+    "live_data"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,