SSI: Verification of metrics fails intermittently

This is a failure within the check_stats routine.
http://jenkins.lotus.hpdd.lab.intel.com/job/integration-tests-shared-storage-configuration/5482/
(tests.integration.shared_storage_configuration.test_managed_filesystem_with_failover.TestManagedFilesystemWithFailover) ... FAIL
FAIL
Traceback (most recent call last):
  File "/usr/share/chroma-manager/tests/integration/shared_storage_configuration/test_managed_filesystem_with_failover.py", line 72, in test_create_filesystem_with_failover_mds
    filesystem_id, volumes_expected_hosts_in_normal_state = self._test_create_filesystem_with_failover()
  File "/usr/share/chroma-manager/tests/integration/shared_storage_configuration/test_managed_filesystem_with_failover.py", line 36, in _test_create_filesystem_with_failover
    self.check_stats(filesystem_id)
  File "/usr/share/chroma-manager/tests/integration/core/stats_testcase_mixin.py", line 154, in check_stats
    'Error in verifying expected MDT metric names being returned from API')
  File "/usr/share/chroma-manager/tests/integration/core/utility_testcase.py", line 147, in wait_until_true
    'Timed out waiting for %s\nError Message %s' % (inspect.getsource(lambda_expression), error_message))
AssertionError: Timed out waiting for         self.wait_until_true(lambda: self._compare_target_metric_names('OST', self.ost_stats, filesystem_id),
                             'Error in verifying expected MDT metric names being returned from API')

Error Message Error in verifying expected MDT metric names being returned from API

Further characterization of the issue:

It's actually the OST query that's having an issue, there is a bug where it claims to be verifying MDT although it's not.

The root cause is the stats being displayed via api are missing a key being requested for OSTs. To explain further:

The test requests these stats:
ost_stats = [
  'filesfree',
  'filestotal',
  'kbytesavail',
  'kbytesfree',
  'kbytestotal',
  'num_exports',
  'stats_connect',
  'stats_create',
  'stats_read_bytes',
  'stats_read_iops',
  'stats_write_bytes',
  'stats_write_iops',
  'tot_dirty',
  'tot_granted',
  'tot_pending'
]

This translates to the following API call:
api/target/metric/?metrics=filesfree,filestotal,kbytesavail,kbytesfree,kbytestotal,num_exports,stats_connect,stats_create,stats_read_bytes,stats_read_iops,stats_write_bytes,stats_write_iops,tot_dirty,tot_granted,tot_pending&kind=OST&latest=true&format=json

The stats returned look like this:

{
  "3": [
    {
      "data": {
        "filesfree": 2393502,
        "filestotal": 2393659,
        "kbytesavail": 10012032,
        "kbytesfree": 10014080,
        "kbytestotal": 10015616,
        "num_exports": 0,
        "stats_connect": 1,
        "stats_read_bytes": 0,
        "stats_read_iops": 0,
        "stats_write_bytes": 0,
        "stats_write_iops": 0,
        "tot_dirty": 0,
        "tot_granted": 40960000,
        "tot_pending": 0
      },
      "ts": "2018-05-21T01:02:10+00:00"
    }
  ],
  "4": [
    {
      "data": {
        "filesfree": 655103,
        "filestotal": 655360,
        "kbytesavail": 7418532,
        "kbytesfree": 7959204,
        "kbytestotal": 7997248,
        "num_exports": 0,
        "stats_connect": 3,
        "stats_create": 2,
        "stats_read_bytes": 0,
        "stats_read_iops": 0,
        "stats_write_bytes": 5120000,
        "stats_write_iops": 10,
        "tot_dirty": 0,
        "tot_granted": 279104,
        "tot_pending": 0
      },
      "ts": "2018-05-21T01:02:30+00:00"
    }
  ]
}

The root issue is that stats_create is missing from one of the targets, and it's being expected.

that kind of makes sense given data is only being written to one of the osts, making sure the data get striped might be a good way to fix this

that kind of makes sense given data is only being written to one of the osts, making sure the data get striped might be a good way to fix this

My thoughts as well. Going to test with a smaller file to see if I can recreate.

Client commands I've used in the past to do this if useful:
19  mount -t lustre <IP_ADDRESS>@tcp:<IP_ADDRESS>@tcp:/fstest /mnt/fstest
20  dd if=/dev/zero of=testf1.txt bs=3M count=2
29  lfs setstripe -c 2 -S 512k /mnt/fstest
30  lfs getstripe /mnt/fstest/
31  cp testf1.txt /mnt/fstest/

stats_create is a RPC counter and is not reliable for data object creation.
Going to remove it from collection, we don't appear to use it or stats_connect anywhere else in IML.


oak will not compile on nightly (rustc 1.32.0-nightly (653da4fd0 2018-11-08))

Oak master fails to build using latest nightly rust: rustc 1.32.0-nightly (653da4fd0 2018-11-08)
After some investigation I was able to solve some of the errors.

syntax::codemap is now renamed as syntax::source_map;
instantiation of rust::Mod @ src/liboak/back/compiler/grammar.rs:74:37 now requires the new field inline which can be initalized as false unless otherwise desired.
syntax::util::small_vector::SmallVector has been moved out of libsyntax. Supposedly it has been moved to librustc_data_structures as per this issue. But I was unable get oak to compile properly.

➜  oak git:(master) ✗ cargo build
Compiling oak v0.5.8 (/Users/alexanderzywicki/code/oak)
error[E0432]: unresolved import syntax::util::small_vector
--> src/liboak/rust.rs:21:23
|
21 | pub use syntax::util::small_vector::SmallVector;
|                       ^^^^^^^^^^^^ Could not find small_vector in util
error[E0432]: unresolved import syntax::codemap
--> src/liboak/rust.rs:22:17
|
22 | pub use syntax::codemap::{DUMMY_SP, NO_EXPANSION, Span, MultiSpan, Spanned, respan, BytePos};
|                 ^^^^^^^ Could not find codemap in syntax
error[E0433]: failed to resolve. Use of undeclared type or module Span
--> src/liboak/front/ast.rs:72:13
|
72 |       span: Span::new(lo,hi,NO_EXPANSION)
|             ^^^^ Use of undeclared type or module Span
error[E0433]: failed to resolve. Use of undeclared type or module Span
--> src/liboak/front/parser.rs:225:9
|
225 |         Span::new(lo, hi, NO_EXPANSION),
|         ^^^^ Use of undeclared type or module Span
error[E0412]: cannot find type Span in this scope
--> src/liboak/ast.rs:82:50
|
82 |   pub fn multi_locations_err(&self, errors: Vec<(Span, String)>) {
|                                                  ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
17 | use syntax::ext::quote::rt::Span;
|
17 | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/ast.rs:94:53
|
94 |   pub fn multi_locations_warn(&self, warnings: Vec<(Span, String)>) {
|                                                     ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
17 | use syntax::ext::quote::rt::Span;
|
17 | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/ast.rs:100:32
|
100 |   pub fn span_err(&self, span: Span, msg: String) {
|                                ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
17  | use syntax::ext::quote::rt::Span;
|
17  | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/ast.rs:104:33
|
104 |   pub fn span_note(&self, span: Span, msg: String) {
|                                 ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
17  | use syntax::ext::quote::rt::Span;
|
17  | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/ast.rs:234:21
|
234 |   fn span(&self) -> Span {
|                     ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
17  | use syntax::ext::quote::rt::Span;
|
17  | use syntax::source_map::Span;
|
error[E0412]: cannot find type BytePos in this scope
--> src/liboak/front/ast.rs:41:36
|
41 |   pub fn alloc_expr(&mut self, lo: BytePos, hi: BytePos, expr: Expression) -> usize {
|                                    ^^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::BytePos;
|
15 | use syntax::source_map::BytePos;
|
error[E0412]: cannot find type BytePos in this scope
--> src/liboak/front/ast.rs:41:49
|
41 |   pub fn alloc_expr(&mut self, lo: BytePos, hi: BytePos, expr: Expression) -> usize {
|                                                 ^^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::BytePos;
|
15 | use syntax::source_map::BytePos;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/front/ast.rs:65:13
|
65 |   pub span: Span
|             ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::Span;
|
15 | use syntax::source_map::Span;
|
error[E0412]: cannot find type BytePos in this scope
--> src/liboak/front/ast.rs:70:18
|
70 |   fn spanned(lo: BytePos, hi: BytePos) -> FExpressionInfo {
|                  ^^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::BytePos;
|
15 | use syntax::source_map::BytePos;
|
error[E0412]: cannot find type BytePos in this scope
--> src/liboak/front/ast.rs:70:31
|
70 |   fn spanned(lo: BytePos, hi: BytePos) -> FExpressionInfo {
|                               ^^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::BytePos;
|
15 | use syntax::source_map::BytePos;
|
error[E0425]: cannot find value NO_EXPANSION in this scope
--> src/liboak/front/ast.rs:72:29
|
72 |       span: Span::new(lo,hi,NO_EXPANSION)
|                             ^^^^^^^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
15 | use syntax::source_map::NO_EXPANSION;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/front/ast.rs:78:21
|
78 |   fn span(&self) -> Span {
|                     ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::Span;
|
15 | use syntax::source_map::Span;
|
error[E0412]: cannot find type BytePos in this scope
--> src/liboak/front/parser.rs:42:32
|
42 |   fn alloc_expr(&mut self, lo: BytePos, hi: BytePos, expr: Expression) -> usize {
|                                ^^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::BytePos;
|
15 | use syntax::source_map::BytePos;
|
error[E0412]: cannot find type BytePos in this scope
--> src/liboak/front/parser.rs:42:45
|
42 |   fn alloc_expr(&mut self, lo: BytePos, hi: BytePos, expr: Expression) -> usize {
|                                             ^^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::BytePos;
|
15 | use syntax::source_map::BytePos;
|
error[E0412]: cannot find type BytePos in this scope
--> src/liboak/front/parser.rs:171:32
|
171 |   fn parse_type(&mut self, lo: BytePos, expr: usize, rule_name: &str) -> rust::PResult<'a, usize> {
|                                ^^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15  | use syntax::ext::quote::rt::BytePos;
|
15  | use syntax::source_map::BytePos;
|
error[E0425]: cannot find value NO_EXPANSION in this scope
--> src/liboak/front/parser.rs:225:27
|
225 |         Span::new(lo, hi, NO_EXPANSION),
|                           ^^^^^^^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
15  | use syntax::source_map::NO_EXPANSION;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/middle/typing/ast.rs:107:13
|
107 |   pub span: Span,
|             ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
18  | use syntax::ext::quote::rt::Span;
|
18  | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/middle/typing/ast.rs:112:21
|
112 |   fn span(&self) -> Span {
|                     ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
18  | use syntax::ext::quote::rt::Span;
|
18  | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/middle/typing/ast.rs:120:18
|
120 |   pub fn new(sp: Span, ty: Ty) -> Self {
|                  ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
18  | use syntax::ext::quote::rt::Span;
|
18  | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/middle/typing/ast.rs:140:20
|
140 |   pub fn infer(sp: Span) -> Self {
|                    ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
18  | use syntax::ext::quote::rt::Span;
|
18  | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/back/context.rs:250:47
|
250 |   pub fn expr_span(&self, expr_idx: usize) -> Span {
|                                               ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
16  | use syntax::ext::quote::rt::Span;
|
16  | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/back/code_printer.rs:38:83
|
38 | fn print_module(s: &mut State, module: &Mod, ident: Ident, vis: Visibility, span: Span)
|                                                                                ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::Span;
|
15 | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/back/compiler/value.rs:20:40
|
20 | pub fn tuple_value(cx: &ExtCtxt, span: Span, vars_names: Vec) -> RExpr
|                                        ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
17 | use syntax::ext::quote::rt::Span;
|
17 | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/back/compiler/value.rs:36:42
|
36 | pub fn tuple_pattern(cx: &ExtCtxt, span: Span, vars_names: Vec) -> RPat
|                                          ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
17 | use syntax::ext::quote::rt::Span;
|
17 | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/identifier.rs:39:21
|
39 |   fn span(&self) -> Span;
|                     ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::Span;
|
15 | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/identifier.rs:51:21
|
51 |   fn span(&self) -> Span {
|                     ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::Span;
|
15 | use syntax::source_map::Span;
|
error[E0412]: cannot find type Span in this scope
--> src/liboak/identifier.rs:65:21
|
65 |   fn span(&self) -> Span {
|                     ^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
|
15 | use syntax::ext::quote::rt::Span;
|
15 | use syntax::source_map::Span;
|
error[E0063]: missing field inline in initializer of syntax::ast::Mod
--> src/liboak/back/compiler/grammar.rs:74:37
|
74 |           node: rust::ItemKind::Mod(rust::Mod{
|                                     ^^^^^^^^^ missing inline
error: aborting due to 32 previous errors
Some errors occurred: E0063, E0412, E0425, E0432, E0433.
For more information about an error, try rustc --explain E0063.
error: Could not compile oak.
To learn more, run the command again with --verbose.

Thank you for the issue and the pull request ;-)


Working with CI

I'm currently using autorebase in one of my projects building with CircleCI.
The problem is that autorebase (through its dependencies) creates temporary branch in order to rebase. During the build, Circle tries to checkout this branch, and fails because it has been already removed.
My point is : can we imagine a workaround which lets us to choose the temporary branch name in order to be able to ignore it on CI tools ?
Today, temporary branch name seems to be based on these two lines :

https://github.com/tibdex/github-cherry-pick/blob/master/src/index.js#L256
https://github.com/tibdex/shared-github-internals/blob/master/src/git.js#L22

Hi Guillaume, I use CircleCI too in most of my projects and I always enable the option to only build pull requests. Then CircleCI completely ignores the temporary branches created by Autorebase dependencies. Have you considered enabling this option too?

@Nigui @tibdex Hi both, we had this problem and just so you're aware you can also ignore branches in CircleCI for example:
filters:
  branches:
    ignore:
      # ignore branches created by autorebase https://github.com/tibdex/autorebase
      - /rebase-pull-request-.*/
      - /cherry-pick-rebase-pull-request-.*/

https://circleci.com/docs/2.0/configuration-reference/#branches


Measure duration of experiments

Added the measurement of the experiments and added it to the ExperimentResult and the Measurement.
Included a test to ensure this is populated correctly.

Ack! GitHub Selfie is not working so you'll have to settle for an oldie but goodie.

Thanks! :sparkles:


Feature Request: Add NVM-parameter in config.yaml

Motivation
It is not related to a problem, but it is about to make developer and team life easier when we have the frustrating to install NVM by manually.
As we can see, there are two options today:

Use Dockerfile, but it is not sensing to use the method, especially, as teams who have different developers and not all developers have the same DDEV version. So, it is hard for the team to maintaining the ARG-part with the docker image and DDEV version.
Install it by post-start with bash, but this is not friendly for the developer who is jumping over projects often since sometimes they need to do ddev pause then back to the project.

Solution/suggestion
Could we add a new parameter in the config.yaml? If it is true, then it installs only one time with other tools such as composer, Drush etc, if not true then it doesn't install the nvm.
nvm: true

Just to respond... the ARG part in a Dockerfile is overridden at build time and uses the actual version of the webimage that ddev is using.  Does that solve your problem? Essentially the ARG part is irrelevant in ARG BASE_IMAGE=drud/ddev-webserver:v1.13.1. We should either document that or see if we can get rid of it.

I'm going to go ahead and close this one for now since you have a path with node and whatever you want in project Dockerfile. Happy to continue the conversation here or elsewhere, and happy to reopen this if you think it's still necessary.


Consistent rate limit errors in HTTPClient.modify_channel

Check #162 for a test case
Logs
INFO     nextcore.http.client.client:client.py:301 Requesting PATCH /channels/1065579508025020506
DEBUG    nextcore.common.times_per.times_per:times_per.py:137 Releasing 0 requests
DEBUG    nextcore.http.client.client:client.py:313 Response status: 429
DEBUG    nextcore.common.dispatcher:dispatcher.py:439 Dispatching event request_response
INFO     nextcore.http.client.client:client.py:356 Exceeded rate-limit on shared route! (None1065579508025020506NoneNonePATCH/channels/{channel_id})This is not unexpected as shared rate-limits are shared by multiple users (and therefore, are out of our control). This does not count towards a cloudflare ban. Retry after: 1.667
DEBUG    nextcore.common.times_per.times_per:times_per.py:86 Calculated remaining: 49
DEBUG    nextcore.common.times_per.times_per:times_per.py:87 Reserved requests: 0
INFO     nextcore.http.client.client:client.py:301 Requesting PATCH /channels/1065579508025020506
DEBUG    nextcore.http.client.client:client.py:313 Response status: 429
DEBUG    nextcore.common.dispatcher:dispatcher.py:439 Dispatching event request_response
INFO     nextcore.http.client.client:client.py:356 Exceeded rate-limit on shared route! (None1065579508025020506NoneNonePATCH/channels/{channel_id})This is not unexpected as shared rate-limits are shared by multiple users (and therefore, are out of our control). This does not count towards a cloudflare ban. Retry after: 1.268
DEBUG    nextcore.common.times_per.times_per:times_per.py:86 Calculated remaining: 48
DEBUG    nextcore.common.times_per.times_per:times_per.py:87 Reserved requests: 0
INFO     nextcore.http.client.client:client.py:301 Requesting PATCH /channels/1065579508025020506
DEBUG    nextcore.http.client.client:client.py:313 Response status: 429
DEBUG    nextcore.common.dispatcher:dispatcher.py:439 Dispatching event request_response
INFO     nextcore.http.client.client:client.py:356 Exceeded rate-limit on shared route! (None1065579508025020506NoneNonePATCH/channels/{channel_id})This is not unexpected as shared rate-limits are shared by multiple users (and therefore, are out of our control). This does not count towards a cloudflare ban. Retry after: 1.044
DEBUG    nextcore.common.times_per.times_per:times_per.py:86 Calculated remaining: 47
DEBUG    nextcore.common.times_per.times_per:times_per.py:87 Reserved requests: 0
INFO     nextcore.http.client.client:client.py:301 Requesting PATCH /channels/1065579508025020506
DEBUG    nextcore.http.client.client:client.py:313 Response status: 429
DEBUG    nextcore.common.dispatcher:dispatcher.py:439 Dispatching event request_response
INFO     nextcore.http.client.client:client.py:356 Exceeded rate-limit on shared route! (None1065579508025020506NoneNonePATCH/channels/{channel_id})This is not unexpected as shared rate-limits are shared by multiple users (and therefore, are out of our control). This does not count towards a cloudflare ban. Retry after: 0.827
DEBUG    nextcore.common.times_per.times_per:times_per.py:86 Calculated remaining: 46
DEBUG    nextcore.common.times_per.times_per:times_per.py:87 Reserved requests: 0
INFO     nextcore.http.client.client:client.py:301 Requesting PATCH /channels/1065579508025020506
DEBUG    nextcore.common.times_per.times_per:times_per.py:137 Releasing 0 requests
DEBUG    nextcore.http.client.client:client.py:313 Response status: 429
DEBUG    nextcore.common.dispatcher:dispatcher.py:439 Dispatching event request_response
INFO     nextcore.http.client.client:client.py:356 Exceeded rate-limit on shared route! (None1065579508025020506NoneNonePATCH/channels/{channel_id})This is not unexpected as shared rate-limits are shared by multiple users (and therefore, are out of our control). This does not count towards a cloudflare ban. Retry after: 0.422
DEBUG    nextcore.common.times_per.times_per:times_per.py:86 Calculated remaining: 49
DEBUG    nextcore.common.times_per.times_per:times_per.py:87 Reserved requests: 0
INFO     nextcore.http.client.client:client.py:301 Requesting PATCH /channels/1065579508025020506
DEBUG    nextcore.http.client.client:client.py:313 Response status: 429
DEBUG    nextcore.common.dispatcher:dispatcher.py:439 Dispatching event request_response
INFO     nextcore.http.client.client:client.py:356 Exceeded rate-limit on shared route! (None1065579508025020506NoneNonePATCH/channels/{channel_id})This is not unexpected as shared rate-limits are shared by multiple users (and therefore, are out of our control). This does not count towards a cloudflare ban. Retry after: 0.085

Possibly a #135 dupe?


Compress images

In side-to-side comparison, I never really see any difference before and after using tinypng and tinyjpg, and I think in the case of screenshots and sample files, there's almost no reason not to. (I don't believe there are any licensing-related issues with this commit either.)
This should reduce the initial size of a Davros grain by about half, and the overall SPK file by like half a megabyte. I don't see much reason to push a release specifically for this minor optimization, but I thought it made sense to offer it up.

And apparently somehow this ended up including the commit to sync my repo with upstream. :/