[WIP DO NOT MERGE] Add more states for attaching disk in allocated state

With this patch https://github.com/apache/cloudstack/commit/b766bf7fc9787b2516eee2eac6f00dd03a327b8b
we started tracking disks in attaching state so that other attach request can fail gracefully. However this missed the case where disks were in allocated state but attach was requested.
For the use case where users want to attach disk in allocated state but not ready, we need to have more states for keeping track of attaching state.
Test this by creating a VM with "startvm=false", create a disk and try attaching it in allocated state. It would give an exception on latest 4.11 but will be fixed on this patch.
Description

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):
How Has This Been Tested?

Manually (using CloudMonkey, dashboard) and through Marvin Test

@blueorangutan package

@anuragaw a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2701

@blueorangutan test

@anuragaw a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3496)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33075 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3276-t3496-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 67 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80
Failure
416.00
test_internal_lb.py

@anuragaw can you implement a marvin test for the test-case.

@blueorangutan package

@anuragaw a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2704

@anuragaw can you implement a marvin test for the test-case.

Done @rhtyd , let me know if this covers the case . Thanks!

@blueorangutan package

@anuragaw a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2710

@blueorangutan test

@anuragaw a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan package

@anuragaw a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2711

@blueorangutan test

@anuragaw a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan package

@anuragaw a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2712

Trillian test result (tid-3504)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29605 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3276-t3504-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 68 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-3506)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29936 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3276-t3506-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Smoke tests completed. 68 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2715

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos6 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-3525)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 26397 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3276-t3525-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 68 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-3524)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 27352 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3276-t3524-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Smoke tests completed. 67 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_scale_vm
Error
17.52
test_scale_vm.py


Docker containers for testing, teaching, demonstrations and as a basis for new projects

Docker seems to be potentially very useful, for:

testing (e.g. creating throwaway Postgres or Sumatra-Server installations)
teaching/demonstrations (when giving tutorials, simplifies installation)
providing a basis for new projects. In the interests of reproducible research, performing all your research in Docker containers will definitely improve reproducibility and ease of sharing.

Probably we should provide build scripts ("Dockerfiles") in the Sumatra repository, and create an account on the Docker index to host the images.
Imported from Bitbucket issue 197,  created by apdavison on 03-25-2014 at 10:48, last modified: 03-25-2014 at 11:20

also see Docker images from Sheila Miguez: https://index.docker.io/u/codersquid/sumatra-tip/
Trac comment by apdavison on 03-25-2014 at 11:20


Some wrong with order where and whereIn ..

I'm using laravel framework with
"php": ">=7.0.0",
"laravel/framework": "5.5.*",
I'm try to query with

$locations = \DB::table('locations')
->where('loc_level',$level)
->whereIn('id', $arrId)
->get();
2.1. $locations = \DB::table('locations')
->whereIn('id', $arrId)
->where('loc_level',$level)
->get();

I have got command query on debug bar. It's not same..
In case one. The values is have change. It have destroy a some value.
In case two. It's ok..
Please check again this._

Can you provide the values for $arrId and $level, and the generated sql statements?

This is value of that variables
$arrId = [40, 1, 2, 3, 4, 7, 8, 46];
$level = 1;
...

Hi, can you post your error message ?

I've tried this on a fresh Laravel 5.7 install and I cant replicate.
If you have more information please feel free to post it here and I'll reopen the ticket.


Adds a new "migrating from 1.x to 2.x" section to the documentation

Just left a few individual comments (don't know why I didn't go with a review instead) on a few details that were missing, but the guide is great work @ajsantander!! Kudos :+1:


Cargo 0.28.0 and later break cargo-bloat for embedded

As reported in https://github.com/RazrFalcon/cargo-bloat/issues/25 I upgraded cargo-bloat to the latest version which also bumps the cargo dependency to 0.28.0 and this seemingly breaks cargo bloat for use on cross-compiled binaries. I verified that bumping the dependency to 0.29.0 does not address the issue while going back to 0.27.0 rectifies the problem.

Thanks for the report! We don't provide stability guarantees though about Cargo used through its library API. To that end we're happy to take PRs but we don't track the issues like this here unless it's a bug with Cargo-the-CLI

So this is it then? A valuable tool breaks, the author can't fix the problem because it's in a third party component and the authors of a third party component won't even look at the problem?
Frustrating...

As a solution going forward, perhaps it's worth having a CI build (either during PR validation and/or a nightly build) that tests against latest/source cargo? The idea being that you would have forewarning when a breaking change lands.
Maybe it should be a best practice for cargo extensions (custom commands, or whatnot).

It's certainly a good idea to do that but I don't see how it addresses the problem. Say you're forewarned about an upcoming compatibility issue, what are the options? Staying on the old version indefinitely? And then let me ask the same question again but with the background that the tool we're talking about just so happened to work for embedded development without any intention to enable that use case in the first place.


Soft limit for maxEventCount

Summary
When you have e.g. 4 events per day, set maxDaysToShow: 2 and maxEventCount: 6 you'll have only half of tomorrows events listed. If you could set maxEventSoftCount: 6, tomorrow would either not shown or shown completely (depending on the way the function would be implemented)

Motivation
This is especially relevant if you have multiple calendars in the card or lots of events per day in one calendar. With the current hard limit, it can give the impression, that you only have e.g. 2 events for the last day shown in the calendar card while instead you have e.g. 4 events. If you look on the calendar card to plan something for the last day shown, you could miss an already planned event.

Describe alternatives you've considered
Instead of another limit setting, there could be a soft-limit-flag:
maxEventCount: 6
maxEventSoft: true

Additional context

Hi,
I don't understand why you would want to have a limit of 6 events but then see all the events for a day? If you want to see all the events for the two days you wouldn't set the maxEventCount
Or am I missunderstanding why you would want this feature?

Hi,
maybe my example was bad - I thought it would be more relevant to people with many events per day.
I'll try to explain how I'd use it (and was using with "ljmerza/calendar-card"):
I have a calendar card on my dashboard which listed around 10 calendar entries all the time. Sometimes I have e.g. 3 events a day and other times I have only two events in a week. If I only set e.g. maxDaysToShow: 7 that calendar card would sometimes be very long and you'd have to scroll far down to see the other cards - or sometimes half of the dashboard would be empty.
On the other hand if I set maxEventCount: 10, I'd never know if all events of the last day are shown in the calendar card.
If there's a soft limit, the length of the calendar card wouldn't vary that much, but there wouldn't be that "useless" incomplete last day in the card. So it's kind of a compromise between those settings.

I've tried to understand this however I'm still not sure what you mean.
If I have say maxDaysToShow: 7 and maxEventCount: 10 with maxEventSoft: true and say the soft limit shows all the events for the day, you would still have a long list as it would be the equivalent of not setting maxEventCount
I think a better solution would be that if maxEventCount: 10 is set then a new text appears at the bottom saying "x events are not listed" then on clicking it opens the calendar, or something similar.

I've tried to understand this however I'm still not sure what you mean.
If I have say maxDaysToShow: 7 and maxEventCount: 10 with maxEventSoft: true and say the soft limit shows all the events for the day, you would still have a long list as it would be the equivalent of not setting maxEventCount
I think a better solution would be that if maxEventCount: 10 is set then a new text appears at the bottom saying "x events are not listed" then on clicking it opens the calendar, or something similar.

If I have say maxDaysToShow: 7 and maxEventCount: 10 with maxEventSoft: true and say the soft limit shows all the events for the day, you would still have a long list as it would be the equivalent of not setting maxEventCount

Example A) If you have many events on one day, the list would indeed get very long:

maxEventCount: 5SoftLimit: off
maxEventCount: 10SoftLimit: on

Event 1Event 2
Event 1Event 2

Event 3Event 4
Event 3Event 4

Event 5
Event 5Event 6Event 7Event 8Event 9 Event 10 Event 11 Event 12 Event 13 Event 14 Event 15

Example B) If you have only a few events per day, it doesn't make a big difference:

maxEventCount: 5SoftLimit: off
maxEventCount: 10SoftLimit: on

Event 1Event 2
Event 1Event 2

Event 3Event 4
Event 3Event 4

Event 5
Event 5Event 6

I think a better solution would be that if maxEventCount: 10 is set then a new text appears at the bottom saying "x events are not listed" then on clicking it opens the calendar, or something similar.

I think for Example A, your idea would be the perfect solution - the list doesn't get longer but you now see that X events are missing
If like in Example B only one or two events are missing it would be better to show the events directly, in my opinion (so you would not have to click a link to only show one missing event).
To sum it up: For me, the perfect solution would be if a soft limit option exists and the "missing events counter" would be shown if softLimit is disabled - but of course I understand that your time is limited and not every feature can be implemented just because some random dude on the internet wants it ;) (I used the now archived ljmerza/calendar-card before which had that option and it's always quite an adjustment if you got used to something) I already tried to do it myself, so I can open a pull request myself, but sadly I'm to dumb to understand the code...
TL;DR: Your idea would solve the problem I explained in "Motivation" of this issue, so it is indeed a good solution 👍

If I have say maxDaysToShow: 7 and maxEventCount: 10 with maxEventSoft: true and say the soft limit shows all the events for the day, you would still have a long list as it would be the equivalent of not setting maxEventCount

Example A) If you have many events on one day, the list would indeed get very long:

maxEventCount: 5SoftLimit: off
maxEventCount: 10SoftLimit: on

Event 1Event 2
Event 1Event 2

Event 3Event 4
Event 3Event 4

Event 5
Event 5Event 6Event 7Event 8Event 9 Event 10 Event 11 Event 12 Event 13 Event 14 Event 15

Example B) If you have only a few events per day, it doesn't make a big difference:

maxEventCount: 5SoftLimit: off
maxEventCount: 10SoftLimit: on

Event 1Event 2
Event 1Event 2

Event 3Event 4
Event 3Event 4

Event 5
Event 5Event 6

I think a better solution would be that if maxEventCount: 10 is set then a new text appears at the bottom saying "x events are not listed" then on clicking it opens the calendar, or something similar.

I think for Example A, your idea would be the perfect solution - the list doesn't get longer but you now see that X events are missing
If like in Example B only one or two events are missing it would be better to show the events directly, in my opinion (so you would not have to click a link to only show one missing event).
To sum it up: For me, the perfect solution would be if a soft limit option exists and the "missing events counter" would be shown if softLimit is disabled - but of course I understand that your time is limited and not every feature can be implemented just because some random dude on the internet wants it ;) (I used the now archived ljmerza/calendar-card before which had that option and it's always quite an adjustment if you got used to something) I already tried to do it myself, so I can open a pull request myself, but sadly I'm to dumb to understand the code...
TL;DR: Your idea would solve the problem I explained in "Motivation" of this issue, so it is indeed a good solution 👍

I also find JS a headache, I’m mainly a backend developer so it stumps me quite a bit!
Ok I think I understand now and also took a quick look at the other card you mentioned.
So I think I will add two new options:
softLimit: 3 This would then show 3 events over the limit if 3 or less exist, in a case where more exist a link will be shown.
Where no softLimit is set then it will just show a link and count of missing events.
I will see what I can do :)

I also find JS a headache, I’m mainly a backend developer so it stumps me quite a bit!
Ok I think I understand now and also took a quick look at the other card you mentioned.
So I think I will add two new options:
softLimit: 3 This would then show 3 events over the limit if 3 or less exist, in a case where more exist a link will be shown.
Where no softLimit is set then it will just show a link and count of missing events.
I will see what I can do :)

A non-binary softLimit is an great idea  👍👍👍 Thank you very much! :)

A non-binary softLimit is an great idea  👍👍👍 Thank you very much! :)

In testing I have the following:
maxDaysToShow: 7
maxEventCount: 3

maxDaysToShow: 7
maxEventCount: 3
softLimit: 1

maxDaysToShow: 7
maxEventCount: 6
softLimit: 1

maxDaysToShow: 7
maxEventCount: 6
softLimit: 3

In testing I have the following:
maxDaysToShow: 7
maxEventCount: 3

maxDaysToShow: 7
maxEventCount: 3
softLimit: 1

maxDaysToShow: 7
maxEventCount: 6
softLimit: 1

maxDaysToShow: 7
maxEventCount: 6
softLimit: 3

That's amazing! Thank you very much! :)

That's amazing! Thank you very much! :)


bayes-by-backprop-gluon has some error

There is error in Complet training loop cell
---> 22                 param._data[list(param._data.keys())[0]] = sample
23
24             # forward-propagate the batch
AttributeError: 'list' object has no attribute 'keys'

any fix this issue?


Prisma client hangs when using Graphql Yoga server on lambda

Bug description
We're using graphql yoga 1.18.3 and deploying via the most recent serverless framework to aws lambda:
serverless.yml:
functions:
  graphql:
    handler: src/functions/server.server
    events:
      - http:
          path: /
          method: post
          cors: true

server.ts:
import { GraphQLServerLambda } from 'graphql-yoga';
import { typeDefs } from '../schema';
import resolvers from '../resolvers';
import permissions from '../permissions';
import { createContext } from '../context';

const lambda = new GraphQLServerLambda({
  typeDefs,
  resolvers,
  middlewares: [permissions],
  context: createContext
});

exports.server = lambda.graphqlHandler;
exports.playground = lambda.playgroundHandler;

context.ts:
import { PrismaClient } from '@prisma/client';

const prisma = new PrismaClient();

export interface Context {
  req?: any;
  prisma: PrismaClient;
}

export function createContext(req): Context {
  return { ...req, prisma };
}

Using this generator block in schema.prisma:
generator client {
  provider      = "prisma-client-js"
  binaryTargets = ["native", "rhel-openssl-1.0.x"]
}

We're connecting to a large aurora mysql 5.7 db.
The client call to the db works, but the server then "hangs" after the resolver, consistently making the lambda last 6 seconds before returning a 502 internal server error.
How to reproduce
1.) Setup a quick sample graphql yoga server and deploy it to aws lambda behind an http endpoint.
2.) Create a simple mutation resolver that uses the prisma client.
3.) Access the graphql playground by putting the http endpoint into your browser.
4.) Try the mutation and note the lag and 502 response.
Expected behavior
The function should return quickly and without a 502 internal server error. The only workaround was to run prisma.disconnect() during or right after every resolver. I tried to create a middleware but the "hang" happened before that could be run.
We already had a function that wrapped around each resolver, so I was able to add the disconnect there. I have no idea if that is helpful, but it's working for now:
catch.ts:
export const catchResult = fn => {
    return function (...args) {
        // hook prisma instance here so we can use to disconnect below
        const prisma = args[2].prisma;
        return fn(...args)
            .then((result) => {
                // resolver was successful, disconnect from prisma and return result
                prisma.disconnect();
                console.log("Prisma client successfully disconnected.");
                return result;
            }).catch((err) => {
                // resolver threw an error, disconnect from prisma and throw on
                prisma.disconnect();
                console.log("Prisma client successfully disconnected.");
                throw err;
            });
    };
};

Then in the resolver, we import and wrap our function like this:
signupUser: catchResult(async (_, { firstName, lastName, email, password }, ctx) => {
    const validEmail = validateEmail(email);
    if (!validEmail) {
      throw new Error('Invalid email.');
    }

    const validPassword = validatePassword(password);
    if (!validPassword) {
      throw new Error('The password is too short.');
    }

    const duplicateEmail = await ctx.prisma.user.findMany({ where: { email } });
    if (duplicateEmail.length > 0) {
      throw new Error('There is already an account with this email address.');
    }

    const verificationToken = generateResetToken();

    const user = await ctx.prisma.user.create({
      data: {
        firstName,
        lastName,
        email,
        verificationToken,
        password: hashPassword(password),
        supportToken: uuid()
      }
    });

    await newRegistrationEmail(user, verificationToken);

    const token: Token = jwt.sign({ id: user.id, role: user.role }, process.env.JWT_SECRET, {
      expiresIn: '1y'
    });

    return { token, user };
  })

Prisma information
Schema has simple user model:
enum Role {
  MEMBER
  ADMIN
}

model User {
  id                String       @default(cuid()) @id
  createdAt         DateTime     @default(now())
  updatedAt         DateTime     @updatedAt
  //
  status            Status       @default(ACTIVE)
  email             String       @unique
  emailVerified     Boolean      @default(false)
  emailUpdates      Boolean      @default(true)
  //
  firstName         String?
  lastName          String?
  password          String?
  passwordSet       Boolean      @default(false)
  //
  verificationToken String?      @unique
  //
  role              Role         @default(MEMBER)
  memberships       Membership[]
  //
  stripeCustomerId  String?
  supportToken      String?
}

First noticed the error when trying to check if a user with the given email was already in our db:
const duplicateEmail = await ctx.prisma.user.findMany({ where: { email } });
The "hang" happened whether the resolver threw an error or returned successfully.
Environment & setup
Lambda is set to use nodejs12.x in serverless.yml.
"@prisma/client": "^2.0.0-beta.2" in package.json.

Closing as a duplicate of https://github.com/prisma/prisma/issues/1754
Please read: https://github.com/prisma/prisma/issues/2121
Basically, you will need to set callBackWaitForEmptyEventLoop option to false because event loop can't be empty if we have a db connection from a previous lambda execution.
Export the server like so in your code:
exports.server = (event, context, cb) => {
  // Set to false to send the response right away when the callback executes, instead of waiting for the Node.js event loop to be empty.
  context.callbackWaitsForEmptyEventLoop = false;

  return lambda.graphqlHandler(event, context, cb);
};

I can confirm your fix worked! Thank you! 🙏


Some tables' numbers are extracted wrong from pdf

Summary of your issue
Some tables' numbers are extracted wrong from pdf
Environment

[ ] java -version: 1.8.0_144
[ ] tabula-java version: 1.01
[ ] OS and it's version: Windows 10
[ ] Your PDF URL:
problem.pdf

Steps to reproduce:

Convert "problem.pdf" file into csv file.
Some numbers are wrong.
e.g: Line 12 : 3,797 -> 3,777, 5,317 -> 5,337 and there are some more...

Example command:
java -jar tabula-1.0.1-jar-with-dependencies.jar problem.pdf -o problem.csv
Other info:
Also reproduce on:
tabula-py v1.0.0 (python --version`: 3.6.2) with code:
df = read_pdf('problem.pdf')
tabula v1.1.1

This is pretty bad; silently changing numbers is something we really really want to avoid.
Is that the exact PDF (unedited)? In particular, I wouldn't be surprised if the code that deals with RTL text, like Hebrew, would be capable causing this.
(Both the web GUI and the Python bindings are downstream of this repo. If there's a problem, it's here, not in those repos.)

@oryahud I see the same error you're seeing, but it's not just in Tabula. If I open the PDF in Chrome or Evince, the same problem manifests. Unfortunately, there's nothing I can do to help -- the PDF you have is broken.

This PDF indeed contains Hebrew text but when I open it in Chrome or Evince the problem does not manifests.

@oryahud Thanks for the screenshot. I'm a little confused -- I don't see any Hebrew in there, just "a" and "s" and numbers... (אני קורה קצת עברית. )
Can you try selecting and copy pasting the number that says "5317"? When you paste it, is it "5317" or 5337"?

Thanks for your quick response.
The PDF contained Hebrew but I change it to English due to confidential issues, but be sure in the original PDF I encountered the same issue.
You are more then right when I paste it, it become "5337".
Any suggestions ?
Thanks !

Unfortunately, I don't have a lot of suggestions; since the problem exists in lots of PDF readers, the problem is in the PDF itself. Sometimes tools that let you print to PDF can fix problems like this, by re-rendering the PDF; you can try opening the PDF in a PDF reader, then, "printing" it to a PDF to see if that fixes the problem. It could be a problem with the embedded fonts (which generated some warnings when I opened the PDF, but that could be because I don't have common Hebrew fonts) too.

I will try it.
Thanks again for your amazing help !


Crash

Environment

OS: Linux (Ubuntu 14.04)
Erlang/OTP: 19 [erts-8.2] [source-fbd2db2] [64-bit]
EMQ: 2.07

Description
I have about 6 clients constantly sending MQTT messages to the broker. The rate of messages is about 1-2 messages/sec in total. Al of a sudden the broker crashed.
I have attached the crash log. The errors can be found near the end of the crash log and look like:
2017-03-05 00:33:48 =ERROR REPORT====
** Generic server <0.10131.7> terminating
** Last message in was {keepalive,start,38}
** When Server state == {client_state,{esockd_connection,[{ssl_socket,#Port<0.13372>,{sslsocket,{gen_tcp,#Port<0.13372>,tls_connection,undefined},<0.10136.7>}},#Fun<esockd_transport.3.2626899>,[]]},["<IP_ADDRESS>",58,"42932"],{{10,0,0,200},42932},{10,0,0,200},42932,true,running,undefined,#Fun<emqttd_parser.0.27418187>,{proto_state,{{10,0,0,200},42932},#Fun<emqttd_client.0.8804271>,true,<<"00486d88-941c-4df6-a98f-5a7cf8a18587">>,<0.10131.7>,false,4,<<"MQTT">>,undefined,false,undefined,30,1024,<0.5886.0>,undefined,{1488,674028,646574}},[{max_clientid_len,1024},{max_packet_size,65536},{client_idle_timeout,30}],undefined,undefined}
** Reason for termination ==
** {{badmatch,{error,einval}},[{emqttd_keepalive,start,3,[{file,"src/emqttd_keepalive.erl"},{line,33}]},{emqttd_client,handle_info,2,[{file,"src/emqttd_client.erl"},{line,209}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,681}]},{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,250}]}]}
...
I have no idea why this happens.
All help is welcome.
I also see there are regular crashes in  the dashboard plugin. These erros can be found in the beginning of the crash log and look like:
2017-03-05 00:32:15 =SUPERVISOR REPORT====
Supervisor: 'esockd_connection_sup - <0.1050.0>'
Context:    connection_crashed
Reason:     {json_encode,{bad_term,{'EXIT',{badarith,[{emqttd_vm,loads,0,[{file,"src/emqttd_vm.erl"},{line,161}]},{emq_dashboard_overview,node_info,0,[{file,"src/emq_dashboard_overview.erl"},{line,77}]},{rpc,local_call,3,[{file,"rpc.erl"},{line,330}]},{emq_dashboard_overview,'-nodes_info/0-lc$^0/1-0-',2,[{file,"src/emq_dashboard_overview.erl"},{line,73}]},{emq_dashboard_overview,nodes_info,0,[{file,"src/emq_dashboard_overview.erl"},{line,73}]},{emq_dashboard,'-dispatcher/1-fun-3-',4,[{file,"src/emq_dashboard.erl"},{line,52}]},{mochiweb_http,headers,5,[{file,"src/mochiweb_http.erl"},{line,75}]}]}}}}
Offender:   [{pid,<0.1324.0>},{name,connection},{mfargs,{mochiweb_http,start_link,[{emq_dashboard,handle_request,[{state,"/usr/lib/emqttd/lib/emq_dashboard-2.0.7/priv/www",#Fun<emq_dashboard.1.76099874>}]}]}}]
Best regards,
Frank Montyne
byNubian BVBA
crash.log.zip

I forget to mention that I'm using an SSL connection to the broker
Best regards,
Frank Montyne

It looks like a few processes crashed, not the broker. Process crashes are considered normal in erlang, since we rely on built in recovery and self healing mechanisms and let processes crash when they do not behave well.
Can you confirm the maximum packet size you are sending? Is the error reproducible under some conditions reliably? If yes, can you describe how to reproduce this?
The first crash looks like the connection was unable to report its stats for some reason. src/emqttd_client.erl:250 probably returns error which then causes a badmatch in emqttd_keepalive:start/3. This may be because the socket closed or connection died unexpectedly, or might be a bug/unhandled case with some connection handling code.
Regarding the second crash log, is /proc mounted and accessible by your emqttd? Looks like cpu_sup wasn't able to determine the load average for some reason. This crash can be ignored safely. It won't affect the broker's performance or behaviour, only some stats in the dashboard may not be populated.

Hi,
The maximum packet size per message is about 150-200 bytes well below the 64K max packet size.
It's the first time I had this type of crash. The problem is that the emqttd process was stopped. Running "sudo service emqttd status" gave "emqttd is stopped".  Seems like the self healing didn't work.
I installed EMQ using "sudo gdebi emqttd_2.0.7_ubuntu14.4_amd64.deb".
/proc is mounted.
I tried "sudo -u emqtt cat /proc/cpuinfo" to check if emqtt has access to /proc and that command gave me the cpu information back.

That seems like a bigger issue then. Is the crash log you posted the full one? Anything else you can find in syslogs, maybe? Is emqttd being killed by OS due to resource scarcity?
For anyone else looking at this, looks like {error, einval} is from Conn:getopt([recv_oct]). Digging into esockd and down shows the ultimate calls are to either inet:getopt/2 or ssl:getopt/2, and the error is due to an invalid argument. Since the options [recv_oct] is valid, it means the connection socket is bad. Stale socket, maybe?
@frank-montyne Can you help reproduce this issue? I'm running 2.0.7 since a couple of weeks and haven't seen this issue.

I'll see if I can reproduce the crash.
I uploaded a recent console and error log. I see that there are sometimes warnings and errors. Perhaps it is related to these?
console&error-log.zip

Hi,
The emqtt process crashed again last night. I uploaded my emq.conf and some more log files. The console log contains information at the INFO level. It gives you a better idea of the data packets that are coming in. I'm constantly running EMQ with:
log.console.level = info
because I want a complete trace of what's going on, that should not be a problem I guess?
logs-and-conf.zip

Hi @frank-montyne, I checked all the log files you attached. No hints for the crash. If the Erlang VM crashed, there will be a log/crash.dump file generated. I guess the Erlang process is killed by OS. Please check the syslog at the same time.

Hi,
I don't see a crash.dump file. There is also nothing to see in the syslog files> Nevertheless the emqtt proces was stopped! The weird thing is it seems to stop exactly at 1 AM each time.

Does the server restart at 1? Anything in cron? Anything that may trigger
an oom at 1?

There is nothing specifically scheduled at 1 PM that  I know of. We used to run mosquitto (until last week) on that server and we never had any problems. I installed EMQ using gdebi that's all.
I added the syslogs. Nothing special except for some manual restarts from me.
Crontab contains the following entries:
/etc/crontab: system-wide crontab
Unlike any other crontab you don't have to run the `crontab'
command to install the new version when you edit this file
and files in /etc/cron.d. These files also have username fields,
that none of the other crontabs do.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
m h dom mon dow user  command
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )

syslog.zip

I looked again for a crash.dump file on the whole disk but I didn't find any.
sudo find / -name crash only list the following:
/usr/share/doc/apport/crashdb-conf.txt.gz
/usr/share/apport/kernel_crashdump
/usr/share/apport/testsuite/test_signal_crashes.pyc
/usr/share/apport/testsuite/test_signal_crashes.py
/usr/share/apport/testsuite/test_java_crashes.py
/usr/share/apport/testsuite/test_java_crashes.pyc
/usr/share/apport/testsuite/crash.class
/usr/share/apport/testsuite/crash.jar
/usr/share/apport/testsuite/test_crash_digger.pyc
/usr/share/apport/testsuite/test_crashdb.py
/usr/share/apport/testsuite/test_crash_digger.py
/usr/share/apport/testsuite/test_python_crashes.py
/usr/share/apport/testsuite/test_python_crashes.pyc
/usr/share/apport/testsuite/test_crashdb.pyc
/usr/lib/python3/dist-packages/apport/crashdb_impl
/usr/lib/python3/dist-packages/apport/pycache/crashdb.cpython-34.pyc
/usr/lib/python3/dist-packages/apport/crashdb.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.pyc
/usr/lib/emqttd/lib/observer-2.1.2/ebin/crashdump_viewer.beam
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer.tool
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer
/usr/lib/emqttd/lib/lager-3.2.2/ebin/lager_crash_log.beam
/usr/lib/emqttd/log/crash.log.3
/usr/lib/emqttd/log/crash.log
/usr/lib/emqttd/log/crash.log.0
/usr/lib/emqttd/log/crash.log.1
/usr/lib/emqttd/log/crash.log.4
/usr/lib/emqttd/log/crash.log.2
/usr/src/linux-headers-3.16.0-31/include/linux/crash_dump.h
/usr/src/linux-headers-3.16.0-31-generic/include/config/crash
/var/crash
/sys/devices/system/cpu/cpu0/crash_notes
/sys/devices/system/cpu/cpu0/crash_notes_size
/sys/kernel/kexec_crash_size
/sys/kernel/kexec_crash_loaded
/etc/apport/crashdb.conf

These are of interest to us:
/usr/lib/emqttd/log/crash.log.3
/usr/lib/emqttd/log/crash.log
/usr/lib/emqttd/log/crash.log.0
/usr/lib/emqttd/log/crash.log.1
/usr/lib/emqttd/log/crash.log.4
/usr/lib/emqttd/log/crash.log.2
Any chance you can share some of them after anonymising any sensitive info?
On Mon 6 Mar, 2017 19:40 Frank Montyne<EMAIL_ADDRESS>wrote:

I looked again for a crash.dump file on the whole disk but I didn't find
any.
sudo find / -name crash only list the following:
/usr/share/doc/apport/crashdb-conf.txt.gz
/usr/share/apport/kernel_crashdump
/usr/share/apport/testsuite/test_signal_crashes.pyc
/usr/share/apport/testsuite/test_signal_crashes.py
/usr/share/apport/testsuite/test_java_crashes.py
/usr/share/apport/testsuite/test_java_crashes.pyc
/usr/share/apport/testsuite/crash.class
/usr/share/apport/testsuite/crash.jar
/usr/share/apport/testsuite/test_crash_digger.pyc
/usr/share/apport/testsuite/test_crashdb.py
/usr/share/apport/testsuite/test_crash_digger.py
/usr/share/apport/testsuite/test_python_crashes.py
/usr/share/apport/testsuite/test_python_crashes.pyc
/usr/share/apport/testsuite/test_crashdb.pyc
/usr/lib/python3/dist-packages/apport/crashdb_impl
/usr/lib/python3/dist-packages/apport/pycache/crashdb.cpython-34.pyc
/usr/lib/python3/dist-packages/apport/crashdb.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.pyc
/usr/lib/emqttd/lib/observer-2.1.2/ebin/crashdump_viewer.beam
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer.tool
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer
/usr/lib/emqttd/lib/lager-3.2.2/ebin/lager_crash_log.beam
/usr/lib/emqttd/log/crash.log.3
/usr/lib/emqttd/log/crash.log
/usr/lib/emqttd/log/crash.log.0
/usr/lib/emqttd/log/crash.log.1
/usr/lib/emqttd/log/crash.log.4
/usr/lib/emqttd/log/crash.log.2
/usr/src/linux-headers-3.16.0-31/include/linux/crash_dump.h
/usr/src/linux-headers-3.16.0-31-generic/include/config/crash
/var/crash
/sys/devices/system/cpu/cpu0/crash_notes
/sys/devices/system/cpu/cpu0/crash_notes_size
/sys/kernel/kexec_crash_size
/sys/kernel/kexec_crash_loaded
/etc/apport/crashdb.conf
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-284405880, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABnL8O7Sw-Z7T1fvPtjmVO4cV7sPJQReks5rjBPvgaJpZM4MTV3K
.

Ignore that, the ones you shared previously are good enough if that or
similar entries are all they have.
On Mon 6 Mar, 2017 21:03 Phani Mahesh<EMAIL_ADDRESS>wrote:

These are of interest to us:
/usr/lib/emqttd/log/crash.log.3
/usr/lib/emqttd/log/crash.log
/usr/lib/emqttd/log/crash.log.0
/usr/lib/emqttd/log/crash.log.1
/usr/lib/emqttd/log/crash.log.4
/usr/lib/emqttd/log/crash.log.2
Any chance you can share some of them after anonymising any sensitive info?
On Mon 6 Mar, 2017 19:40 Frank Montyne<EMAIL_ADDRESS>wrote:
I looked again for a crash.dump file on the whole disk but I didn't find
any.
sudo find / -name crash only list the following:
/usr/share/doc/apport/crashdb-conf.txt.gz
/usr/share/apport/kernel_crashdump
/usr/share/apport/testsuite/test_signal_crashes.pyc
/usr/share/apport/testsuite/test_signal_crashes.py
/usr/share/apport/testsuite/test_java_crashes.py
/usr/share/apport/testsuite/test_java_crashes.pyc
/usr/share/apport/testsuite/crash.class
/usr/share/apport/testsuite/crash.jar
/usr/share/apport/testsuite/test_crash_digger.pyc
/usr/share/apport/testsuite/test_crashdb.py
/usr/share/apport/testsuite/test_crash_digger.py
/usr/share/apport/testsuite/test_python_crashes.py
/usr/share/apport/testsuite/test_python_crashes.pyc
/usr/share/apport/testsuite/test_crashdb.pyc
/usr/lib/python3/dist-packages/apport/crashdb_impl
/usr/lib/python3/dist-packages/apport/pycache/crashdb.cpython-34.pyc
/usr/lib/python3/dist-packages/apport/crashdb.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.pyc
/usr/lib/emqttd/lib/observer-2.1.2/ebin/crashdump_viewer.beam
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer.tool
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer
/usr/lib/emqttd/lib/lager-3.2.2/ebin/lager_crash_log.beam
/usr/lib/emqttd/log/crash.log.3
/usr/lib/emqttd/log/crash.log
/usr/lib/emqttd/log/crash.log.0
/usr/lib/emqttd/log/crash.log.1
/usr/lib/emqttd/log/crash.log.4
/usr/lib/emqttd/log/crash.log.2
/usr/src/linux-headers-3.16.0-31/include/linux/crash_dump.h
/usr/src/linux-headers-3.16.0-31-generic/include/config/crash
/var/crash
/sys/devices/system/cpu/cpu0/crash_notes
/sys/devices/system/cpu/cpu0/crash_notes_size
/sys/kernel/kexec_crash_size
/sys/kernel/kexec_crash_loaded
/etc/apport/crashdb.conf
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-284405880, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABnL8O7Sw-Z7T1fvPtjmVO4cV7sPJQReks5rjBPvgaJpZM4MTV3K
.

Yes that's it.
On Mon, Mar 6, 2017 at 4:35 PM, J Phani Mahesh<EMAIL_ADDRESS>wrote:

Ignore that, the ones you shared previously are good enough if that or
similar entries are all they have.
On Mon 6 Mar, 2017 21:03 Phani Mahesh<EMAIL_ADDRESS>wrote:

These are of interest to us:
/usr/lib/emqttd/log/crash.log.3
/usr/lib/emqttd/log/crash.log
/usr/lib/emqttd/log/crash.log.0
/usr/lib/emqttd/log/crash.log.1
/usr/lib/emqttd/log/crash.log.4
/usr/lib/emqttd/log/crash.log.2
Any chance you can share some of them after anonymising any sensitive
info?
On Mon 6 Mar, 2017 19:40 Frank Montyne<EMAIL_ADDRESS>wrote:
I looked again for a crash.dump file on the whole disk but I didn't find
any.
sudo find / -name crash only list the following:
/usr/share/doc/apport/crashdb-conf.txt.gz
/usr/share/apport/kernel_crashdump
/usr/share/apport/testsuite/test_signal_crashes.pyc
/usr/share/apport/testsuite/test_signal_crashes.py
/usr/share/apport/testsuite/test_java_crashes.py
/usr/share/apport/testsuite/test_java_crashes.pyc
/usr/share/apport/testsuite/crash.class
/usr/share/apport/testsuite/crash.jar
/usr/share/apport/testsuite/test_crash_digger.pyc
/usr/share/apport/testsuite/test_crashdb.py
/usr/share/apport/testsuite/test_crash_digger.py
/usr/share/apport/testsuite/test_python_crashes.py
/usr/share/apport/testsuite/test_python_crashes.pyc
/usr/share/apport/testsuite/test_crashdb.pyc
/usr/lib/python3/dist-packages/apport/crashdb_impl
/usr/lib/python3/dist-packages/apport/pycache/crashdb.cpython-34.pyc
/usr/lib/python3/dist-packages/apport/crashdb.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.py
/usr/lib/python2.7/dist-packages/twisted/test/crash_test_dummy.pyc
/usr/lib/emqttd/lib/observer-2.1.2/ebin/crashdump_viewer.beam
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer.tool
/usr/lib/emqttd/lib/observer-2.1.2/priv/crashdump_viewer
/usr/lib/emqttd/lib/lager-3.2.2/ebin/lager_crash_log.beam
/usr/lib/emqttd/log/crash.log.3
/usr/lib/emqttd/log/crash.log
/usr/lib/emqttd/log/crash.log.0
/usr/lib/emqttd/log/crash.log.1
/usr/lib/emqttd/log/crash.log.4
/usr/lib/emqttd/log/crash.log.2
/usr/src/linux-headers-3.16.0-31/include/linux/crash_dump.h
/usr/src/linux-headers-3.16.0-31-generic/include/config/crash
/var/crash
/sys/devices/system/cpu/cpu0/crash_notes
/sys/devices/system/cpu/cpu0/crash_notes_size
/sys/kernel/kexec_crash_size
/sys/kernel/kexec_crash_loaded
/etc/apport/crashdb.conf
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-284405880, or
mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABnL8O7Sw-
Z7T1fvPtjmVO4cV7sPJQReks5rjBPvgaJpZM4MTV3K>
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-284431274, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJKoLx0AnkGE8SPXdTe2vyDnFGUimdrLks5rjCe_gaJpZM4MTV3K
.

Hi,
This night I had the problem again but slightly different. When I woke up this morning I saw that my service was not receiving any messages anymore from the broker. The broker however was still running. It was receiving messages from our nodes (hardware devices sending MQTT messages to specific topics) but it was not publishing them anymore to my service that is subscribed to these topics. The broker was using 100% cpu. I took some screenshots for you:
The first shows a view with htop where you can see the emqttd process taking up all cpu.
The seconds screenshot is from the dashboard that shows the broker is using a lot of memory (up from around 60 MB under normal circumstances but now at around 400MB). It was still receiving messages since the received count was still going up (314904 at the moment of the screenshot) but only 254494 were already delivered. The sent count did ot go up anymore, it stayed fixed at the 254494 count. So around 60000 still needed to be published (and going up).
The third screenshot is from my service, I restarted the service just to make sure but no messages were coming in. I had to stop the broker process and restart it (losing 60000 messages in the process) before things went back to normal.
screenshots.zip

I'm considering the options on how this can be reproduced. Are you comfortable with erlang? It is possible to inspect the emqttd instance when it reaches this state, and that can be helpful. Specifically, we need to look at erlang process count and if there is a process leak, possibility of a binary leak, and average message queue length of processes and how it varies.
Do you have or can you arrange for trends of cpu, ram and network usage for one night when this happens? The graphs may hold a clue.
My current blind bet: EMQ dispatcher is going wonky, or client/session processes are getting locked up on something.

Hi,
I'm not really comfortable with Erlang. Yesterday I wrote a load tester in
java to try and crash the EMQ broker. I let it run simulating 100 clients
each sending 1000 messages/minute and could not make the broker crash.
Tomorrow (almost midnight in Belgium) we are going to do a test with with
our hardware device and see if that triggers the crash. If that works I can
setup a VM with EMQ on Azure and give you access to it to replicate the
test again. What do you think?
Up until now we were using mosquitto and we never had any real problems. We
processed about 100.000.000 messages in the last year coming from our
hardware.
If you want to know more about what we do you can have a look on our
website (www.bynubian.com)
Best regards,
Frank Montyne
On Wed, Mar 8, 2017 at 8:55 PM, J Phani Mahesh<EMAIL_ADDRESS>wrote:

I'm considering the options on how this can be reproduced. Are you
comfortable with erlang? It is possible to inspect the emqttd instance when
it reaches this state, and that can be helpful. Specifically, we need to
look at erlang process count and if there is a process leak, possibility of
a binary leak, and average message queue length of processes and how it
varies.
Do you have or can you arrange for trends of cpu, ram and network usage
for one night when this happens? The graphs may hold a clue.
My current blind bet: EMQ dispatcher is going wonky, or client/session
processes are getting locked up on something.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-285150243, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJKoL_r0o5D8uJaQZRWAyQCLXZ4dFuc5ks5rjwebgaJpZM4MTV3K
.

To crash emqttd using just number of messages shouldn't be possible
technically. (tm). You can happily increase the load a hundred fold or more
and it should be absorbed. Things may slow down if your hardware is on the
low end, but that should be it.
If this can be replicated in a VM, that would be great. I'll take out some
time to help debug this if I can get my hands on emqttd when it misbehaves.
On Thu 9 Mar, 2017 03:20 Frank Montyne<EMAIL_ADDRESS>wrote:

Hi,
I'm not really comfortable with Erlang. Yesterday I wrote a load tester in
java to try and crash the EMQ broker. I let it run simulating 100 clients
each sending 1000 messages/minute and could not make the broker crash.
Tomorrow (almost midnight in Belgium) we are going to do a test with with
our hardware device and see if that triggers the crash. If that works I can
setup a VM with EMQ on Azure and give you access to it to replicate the
test again. What do you think?
Up until now we were using mosquitto and we never had any real problems. We
processed about 100.000.000 messages in the last year coming from our
hardware.
If you want to know more about what we do you can have a look on our
website (www.bynubian.com)
Best regards,
Frank Montyne
On Wed, Mar 8, 2017 at 8:55 PM, J Phani Mahesh<EMAIL_ADDRESS>wrote:

I'm considering the options on how this can be reproduced. Are you
comfortable with erlang? It is possible to inspect the emqttd instance
when
it reaches this state, and that can be helpful. Specifically, we need to
look at erlang process count and if there is a process leak, possibility
of
a binary leak, and average message queue length of processes and how it
varies.
Do you have or can you arrange for trends of cpu, ram and network usage
for one night when this happens? The graphs may hold a clue.
My current blind bet: EMQ dispatcher is going wonky, or client/session
processes are getting locked up on something.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-285150243, or
mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AJKoL_r0o5D8uJaQZRWAyQCLXZ4dFuc5ks5rjwebgaJpZM4MTV3K
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-285180683, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABnL8CHpaxyi1EDmc5UP_aCjoYxb91D2ks5rjyKKgaJpZM4MTV3K
.

Cool thanks, working on setting something up today.
On Thu, Mar 9, 2017 at 5:54 AM, J Phani Mahesh<EMAIL_ADDRESS>wrote:

To crash emqttd using just number of messages shouldn't be possible
technically. (tm). You can happily increase the load a hundred fold or more
and it should be absorbed. Things may slow down if your hardware is on the
low end, but that should be it.
If this can be replicated in a VM, that would be great. I'll take out some
time to help debug this if I can get my hands on emqttd when it misbehaves.
On Thu 9 Mar, 2017 03:20 Frank Montyne<EMAIL_ADDRESS>wrote:

Hi,
I'm not really comfortable with Erlang. Yesterday I wrote a load tester
in
java to try and crash the EMQ broker. I let it run simulating 100 clients
each sending 1000 messages/minute and could not make the broker crash.
Tomorrow (almost midnight in Belgium) we are going to do a test with with
our hardware device and see if that triggers the crash. If that works I
can
setup a VM with EMQ on Azure and give you access to it to replicate the
test again. What do you think?
Up until now we were using mosquitto and we never had any real problems.
We
processed about 100.000.000 messages in the last year coming from our
hardware.
If you want to know more about what we do you can have a look on our
website (www.bynubian.com)
Best regards,
Frank Montyne
On Wed, Mar 8, 2017 at 8:55 PM, J Phani Mahesh<EMAIL_ADDRESS>wrote:

I'm considering the options on how this can be reproduced. Are you
comfortable with erlang? It is possible to inspect the emqttd instance
when
it reaches this state, and that can be helpful. Specifically, we need
to
look at erlang process count and if there is a process leak,
possibility
of
a binary leak, and average message queue length of processes and how it
varies.
Do you have or can you arrange for trends of cpu, ram and network usage
for one night when this happens? The graphs may hold a clue.
My current blind bet: EMQ dispatcher is going wonky, or client/session
processes are getting locked up on something.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-285150243,
or
mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AJKoL_
r0o5D8uJaQZRWAyQCLXZ4dFuc5ks5rjwebgaJpZM4MTV3K
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-285180683, or
mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABnL8CHpaxyi1EDmc5UP_
aCjoYxb91D2ks5rjyKKgaJpZM4MTV3K>

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/935#issuecomment-285254477, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJKoL7kPcaxHifij3hQvyxogIEBRYdcCks5rj4X8gaJpZM4MTV3K
.

@frank-montyne The benchmark report provided by XMeter: http://emq-xmeter-benchmark-en.readthedocs.io/en/latest/
I still cannot figure out why your deployment crashed when there are still too few clients and messages:(

Hi,
I meanwhile added some logging and did some further tests. Apparently what happened is the following:

At some point in time my persistence service that is subscribed to a number of topics loses its connection to the EMQ broker. The reason why is still unclear.
My service gets notified of the lost connection and starts a background task to reconnect.
For some bizarre reason the reconnect succeeds immediately and I also get a disconnect just afterwards. Then we go into a loop of connects/disconnects. I never had this behaviour with Mosquitto.
Eclipse Paho has an auto reconnect feature itself. I didn't use that up until now because I wanted a bi more control over the reconnect process. I adjusted the code to use that and up until now I had no problems anymore.  Quite weird since it basically does the same thing.

Any ideas about this connect/disconnect loop?
Thanks,
Frank

@frank-montyne ClientId conflicts.

What do you mean? The disconnected client just reconnects with the same client id. Otherwise it would lose its subscriptions.

@frank-montyne At any point of time, if two clients connect with the same client id, the older one gets kicked out. If both of them attempt reconnections immediately, you get the behaviour you describe.
Client id conflicts are disallowed in MQTT specification. I'm sure mosquitto prevents duplicate client ids, most likely following the same behaviour. (An alternate option is to reject the new client)
How are your client ids being generated? Are you letting any framework/library autogenerate them, or are you assigning them using your own scheme? If you rolled your own scheme, what is it? Looking at the logs, you should also see ip and username assosciated with the clients. Or you can trace using emqttd_ctl trace client <client_id> <filename> to see detailed tracess for a particular client id.
Identify the bad client and have its client id changed to fix this issue.

All my clients use a unique UUID (globally unique identifier) that is fixed. This means that if you get a client with the same id it effectively is the same client.  The reason I was reconnecting is because at some point in time Paho notifies me that the connection is broken. At that point I initialize a new Paho client with the same client id (because I don't want to lose my subscriptions) and try to connect again. When I was using Mosquitto that worked without any problems, probably because Mosquitto just cleans up the stale connection.
For EMQ I am now using the autoreconnect feature of Paho instead to avoid getting into trouble.

In that case both mosquitto and EMQ kick the older client if it exists and use the newly connected client for the session. I verified MQTT specifications and this is MUST. All brokers will disconnect existing client and allow newly connected one to take over the session.
When paho notifies that the connection is broken, do you ensure that it is teared down completely before reinitiating connection? If Paho is connecting again automatically, when you create a new connection it can cause the conflict. Is this Paho js or Paho Java? Any example code that you can provide to reproduce the issue?

@frank-montyne Could we close the issue now?

Its Java Paho. Probably the auto reconnect feature of Paho does a better job of cleaning up the stale connection. This issue can be closed. The EMQ broker has been working flawlessly for the last 12 days.
Thanks for the support,
Frank

@frank-montyne Thanks.


Multi GPU with Docker api:2.0cuda11.1

I am using Ubuntu 20.04 with 3 GPUs of RTX 2080. This is my docker-compose:
version: '2.3'

services:
  easynmt:
    image: easynmt/api:2.0-cuda11.1
    restart: always
    runtime: nvidia
    volumes:
      - ./cache:/cache
    ports:
      - "24080:80"
    env_file:
      - ./envfile

The envfile:
EASYNMT_MODEL=m2m_100_1.2B
The first problem is that the service always use only the first GPU in my machine.
The second, one backend worker will take around 7GB of GPU's VRAM, so after a few requests of translate CUDA run out of memory and the code do not work any more (I think the spawned woker need another 7GB VRAM of GPU).
I have my own solution for both: set MAX_WORKERS_BACKEND=1.
start 3 services of easynmt with specificed GPU then use nginx as load balancer to group of 3 easynmt services.
Did I do something wrong? or Any better solution in this case?

Yes, currently Multi-GPU encoding is not supported.
Your setup looks good.


[HW] Bharath

Name: Bharath kadiyala
Email<EMAIL_ADDRESS>Linkedin Profile: https://www.linkedin.com/in/bharath-kadiyala-661556154/

Attach the homework screenshots below for both step II and step III:

You may need to attach "Congratulations!" screenshots from https://www.datastax.com/node/3272 and https://www.datastax.com/dev/scenario/try-it-out-cassandra-query-language-cql as well, @bharathkadiyala1 .

Hello, Bharath, well done! You did the CQL Console part of the homework.
But to complete it, please head to the two short courses as in the 'homework' section of the README, finish them, and post their "congratulations" screenshot. Then we will be able to award you the badge!
Thank you!


Support a custom/specific PPA for Ubuntu systems

+) added $php::repo::ubuntu::ppa
+) conflicts with $php::repo::ubuntu::oldstable

LGTM @fpletz and I'd like to switch to 5.6 soonish by default
btw we should try testing with the php7 ppa soon

Thanks! :+1:


[core] Stop using @types/jss

[x] I have followed (at least) the PR section of the contributing guide.

Breaking changes

Remove the parameter from WithStylesOptions type:

-const options: WithStylesOptions<ClassKey>
+const options: WithStylesOptions;

Closes https://github.com/mui-org/material-ui/issues/14608

I'm wondering could we try more on the @material-ui/styles definitions for the core style modules?

@oliviertassinari

I'm wondering, could we try more on the @material-ui/styles's definitions for the core style modules?

There is certainly something to do. Because, @material-ui/core definitions are not up to date.
Example :
@material-ui/styles
export type StyleRules<Props extends object, ClassKey extends string = string> = Record<
    ClassKey,
    CSSProperties | ((props: Props) => CSSProperties)
  >;

@material-ui/core
export type StyleRules<ClassKey extends string = string> = Record<ClassKey, CSSProperties>;

But why @material-ui/styles function are re-exported in the core module ?

But why @material-ui/styles function are re-exported in the core module?

@VincentLanglet I have tried to expose the motivation in: https://next.material-ui.com/customization/default-theme/#-material-ui-core-styles-vs-material-ui-styles. If it's not clear enough, we should revisit the wording.

@oliviertassinari Ok, if you want default theme, @material-ui/core is enough. If you want to change margin, color, etc, you have to use @material-ui/styles.

@VincentLanglet A JavaScript user without a ThemeProvider should use @material-ui/core. It's what the core components do. This way, people can get started really quickly. A JavaScript user with a ThemeProvider can use @material-ui/core or @material-ui/styles.
A TypeScript users, well, I guess he has to use a wrapped version with his theme?

A JavaScript user with a ThemeProvider can use @material-ui/core or @material-ui/styles.

When you look at the ThemeProvider code in @material-ui/core, it's weird.
import { ThemeProvider } from '@material-ui/styles';

/**
 * @ignore - internal component.
 * @deprecated
 */
export default ThemeProvider;

createMuiTheme can only be import from @material-ui/core
Importing ThemeProvider from material-ui/core is deprecated
=> So if you want to change the theme you have to mix the import.
But then, when you use makeStyle, and everything else, do what you want
=> So the only thing you have to import from @material-ui/styles is the ThemeProvider (cause it's deprecated in the core module).

A TypeScript users, I guess he has to use a wrapped version with his custom theme?

Actually, it seems better to import from @material-ui/styles the types are better.

I think that we should move createMuiTheme() location. This theme object can be used with styled-components and with emotion, it's not coupled with the @material-ui/styles. Maybe?

-import { createMuiTheme } from '@material-ui/core/styles';
+import createMuiTheme from '@material-ui/core/createMuiTheme';

Yes, you have to mix the imports. Do you think that we should keep a ThemeProvider at the core level? Let's say we move to styled-components or emotion, we would have to use their ThemeProvider.

That might be the solution, we encourage people to import from @material-ui/core then they don't have a ThemeProvider, and import from @material-ui/styles as soon as they have one?

I think that we should move the createMuiTheme() module location. This theme object can be used with styled-components and with emotion, it's not coupled with the @material-ui/styles. Maybe?

-import { createMuiTheme } from '@material-ui/core/styles';
+import createMuiTheme from '@material-ui/core/createMuiTheme';

I think it's a great idea

Yes, you have to mix the imports. Do you think that we should keep a ThemeProvider at the core level? Let's say we move to styled-components or emotion, we would have to use their ThemeProvider.
That might be the solution, we encourage people to import from @material-ui/core then they don't have a ThemeProvider, and import from @material-ui/styles as soon as they have one?

No I don't think you should keep a ThemeProvider at the core. Cause indeed, it seems better to encourage people to import from @material-ui/core then they don't have a ThemeProvider, and import from @material-ui/styles as soon as they have one.


Weird dark background in vim terminal buffers

Hi there,
Thanks for the great colorscheme!
I think there is an issue with the dark background for vim terminal buffers:

On the right window, there is a terminal buffer opened with a :terminal command.
It looks greyish, which seems incorrect :)
Any idea what's happening?
Thanks!

Thanks for the suggestion! In my case, :set temguicolors doesn't work (it creates a colorless terminal, see :h xterm-true-color).
But for some reasons the problem disappeared by itself, and the colors of the vim terminal buffer are set correctly now (unrelated to termguicolors). Maybe a commit fixed this since August 2020?
Closing this issue for now!


Standard Library section added to User FAQ

Starting a new section in the User Guide FAQ to cover questions about the relationship between Boost libraries and the Standard Library.

@alandefreitas Good feedback, incorporated it into the answers, and added a few simpler questions.

That looks much better. Thanks!


Support ROCm 6.3

Tasks

[x] Read ROCm Release Notes.
[x] Update AMD driver in Jenkins test infrastructure (ask @kmaehashi).
[ ] Fix code and CI to support the new vesrion.

FlexCI: Update .pfnci/schema.yaml and .pfnci/matirx.yaml. (https://github.com/cupy/cupy/pull/8623)
Wheel Package Detection: Add the package to the duplicate detection. (TBD)

[ ] Backport the above PR.
[ ] Fix cupy-release-tools to support the new version. (https://github.com/cupy/cupy-release-tools/pull/396)
[ ] Backport the above PR.
[ ] Fix documentation.

Add new wheel package to the Installation Guide and README.md.
Update requirements in the installation guide.

[ ] Backport the above PR.
[ ] Add new wheel package to the website.
[ ] Implement or create an issue to support new features, if applicable.

ROCm 6.3 is out.


Browser freezes when using .custom-select or select.form-control

I'm experiencing poor performance when a select element with only couple of options uses .custom-select class.
When I open and close the select, the browser freezes for couple of seconds.

When I remove 'custom-select' selector, it wont freeze.
It freezes only when there is a data table that uses display: grid on the page, although the custom-select is outside the grid's element.

The performance recording in chromium edge shows Event Log

What's going on? it is a bug in chromium, or on bootstrap?
My Environment:

Win10
Chromium Edge Version 89.0.774.54 and Chrome
A reduced test case: https://codepen.io/Lieroo/project/full/XwPeNz

I see no problem in Firefox 86 nor Chromium 89.0.4389.82 on Linux.

Can't reproduce this with the test case you posted. It works without problems on my end. Although my dropdown looks different to yours.

Win10 + Microsoft Edge Version 89.0.774.54

Ok, I think this is related to a specific device. I was able to reproduce it on two high end HP Spectre laptops, but each different model.
I've reported a bug in chromium.


Horizon Lite: Understand request latency characteristics

Can the Lite design support webscale response times? (3-5s)

Measure current latency of /account/* mvp endpoints
Understand history age vs variation of data

Does latency vary with history? provide a distribution, understand the optimal sweet spot
Does latency vary with data variance (account, ledger mix etc)?

Request profiling

Understand current hotspots (indices lookup > txmeta download > request processing etc) and where would potential optimizations be targeted

acceptance criteria - google doc summay

https://docs.google.com/document/d/1hlUbEqD5Jek5YTar9kQ7JvMDLly2wVTr_FdssdPba8c/edit?usp=sharing


Tracking Hindi Localization Progress of Glossary

This issue tracks Hindi localization progress.
Checklist to translate a term

Check that no one else has claimed the page in the comments below.
Comment below the page that you would like to localize, focus on one page per PR, at a time.
Maintainers will add your username and Pull Request next to your selected page in below checklist.
Fork this repo, make your localizations, and submit a pull request following the name convention [hi] [page name from task].
tag/ping a maintainer to make it easier to track progress

Points to remember:

Please use the word localization instead of translation.
For any queries, feel free to contact us in Slack channel
For more information on how to contribute, check out guidelines

For Volunteers
Volunteers can comment below on this issue with the page-name they are interested in localizing.
Please also provide a placeholder PR in your comment.
Example:  I would like to help with/take up/localize page xyz.md . Here's a placeholder PR for the same.
To create a PR:

Fork https://github.com/cncf/glossary/tree/dev-hi repository to your personal account
Create a branch with name suitable for your assigned task e.g. Localize content/hi/style-guide/_index.md
Once the changes are made, create a pull request (PR) against cncf:dev-hi from <your-github-id>:<branch-name>
Mention this issue in the PR description
NOTE: Please create 1 PR for a single page from tasks below. This makes it easier to track progress.

For Maintainers
Volunteers will comment below on this issue with the page-name.
Maintainers will add their username and PR next to the task.
Example : Autoscaling Hindi localization (@contributor-username) #<PR link>
Regularly check for PRs merged and subsequently mark the task done in this issue.
Priorities
We need at least 10 terms in Hindi before go live, below are some suggested terms to work:
Minimum 10 terms required:

[x] config.toml
content/hi/_index.md
content/hi/_TEMPLATE.md
content/hi/cloud_native_tech.md
content/hi/contribute/_index.md
content/hi/style-guide/_index.md
i18n/hi.toml
content/hi/_TEMPLATE.md
@anubha-v-ardhan
#300
[x] content/hi/abstraction.md
@Shubh28698
#355
[ ] content/hi/agile_software_development.md
@anubha-v-ardhan
#377
[ ] content/hi/application_programming_interface.md
@WebShivam
[ ] content/hi/auto_scaling.md
@ShivamTyagi12345
#393
[ ] content/hi/cloud_computing.md
@Shubh28698
[ ] content/hi/cluster.md
@rajatgupta24
#378
[x] content/hi/containerization.md
@verma-kunal
#383
[x] content/hi/devops.md
@anubha-v-ardhan
#381
[ ] content/hi/container.md
@kranurag7
#394
[ ] content/hi/container-image.md
@Garima-Negi
[ ] content/hi/contributor-ladder/_index.md

Additional good to have terms:

[ ] content/hi/api_gateway.md
@ShivamTyagi12345
[ ] content/hi/bare_metal_machine.md
[ ] content/hi/blue_green_deployment.md
[ ] content/hi/canary_deployment.md
[ ] content/hi/client_server_architecture.md
@verma-kunal
[ ] content/hi/cloud_native_apps.md
@anubha-v-ardhan
[ ] content/hi/cloud_native_security.md
@kranurag7
[ ] content/hi/continuous_delivery.md
@verma-kunal
#401
[ ] content/hi/continuous_integration.md
@Aayush987
#396

/language [hi]

@sayantani11 - I can review the PRs and contribute as well

I can review the PRs, but will require someone else as well

I can review the PRs, but will require someone else as well

I can help now that K8s hindi documentation is about to launch, I can find time to review PRs here.

@Garima-Negi I believe, we will also need Site strings i.e. i18n/hi.toml as a minimum requirement. Can you add it to the checklist.

I can review the PRs, but will require someone else as well

I can help with review and approval work too

I would like to assist too.

I can review the PRs, but will require someone else as well

@Garima-Negi Also I think a new language block for Hindi is to be added in config.toml right?

@jayesh-srivastava yes!

@anubha-v-ardhan I guess one page per PR would be great unlike how the Korean team did because then it would be a difficult process to review such huge PRs.

@jayesh-srivastava Not exactly like that. I was saying the initilization of new localization can be done in one PR. i.e just configs.toml and CODEOWNERS. Sorry for miscommunication

I can review the PRs, but will require someone else as well

I can help with review and approval work too

Yeah review and approval

Awesome! I would love to join in as an approver from the next month as well, to help out 🙂

@Garima-Negi I believe we'll also need site strings as minimum requirements, Can we add it to the checklist

content/hi/_TEMPLATE.md
content/hi/_index.md
content/hi/contribute/_index.md
i18n/hi.toml
Solved in #349

I would like to take up content/hi/auto_scaling.md to get started @Garima-Negi

I would like to take up content/hi/auto_scaling.md to get started @Garima-Negi

Hi Shivam, I have updated your name in the list above. Whenever you're ready with a placeholder PR, do share it here or on slack. Thank you!

@Garima-Negi I would like to take content/hi/_TEMPLATE.md

@Shubh28698
content/hi/_TEMPLATE.md
content/hi/_index.md
content/hi/contribute/_index.md
and i18n/hi.toml
have been localized by @sayantani11. see https://github.com/cncf/glossary/issues/270#issuecomment-1019583345
You can take any other file than this

@Shubh28698 content/hi/_TEMPLATE.md content/hi/_index.md content/hi/contribute/_index.md and i18n/hi.toml have been localized by @sayantani11. see #270 (comment) You can take any other file than this. Also check recent comments along with the checklist as the checklist maybe lagging a bit as updating it is a manual work.
@Garima-Negi, can you mark all the above pages on checklist

Sry @anubha-v-ardhan , Then i will take content/hi/abstraction.md

I'm working on content/hi/agile_software_development.md

I'll take content/hi/agile_software_development.md

Hi @Garima-Negi, I would like to take content/hi/application_programming_interface.md to get started with the localization

Hi @Garima-Negi , I would like to take content/hi/cloud_computing.md

@Garima-Negi #349  is closed (ref. https://github.com/cncf/glossary/pull/300#issuecomment-1024894288)
I'll be adding the following pages via #300

config.toml
content/hi/_index.md
content/ko/cloud_native_tech.md
content/hi/contribute/_index.md
content/hi/style-guide/_index.md
i18n/hi.toml

PR #300 will test our config for Hindi localization works. Please mark these pages on the checklist as assigned

@Garima-Negi content/hi/style-guide/_index.md and  content/hi/cloud_native_tech.md are written as separate points too while they are being covered by #300. Please remove them from checklist to avoid confusion.
Also add content/hi/_TEMPLATE.md being covered by #300 itself

Next working on  content/hi/devops.md :)

Hey everyone, I'll be working on content/hi/cluster.md.

A small reminder for everyone, please use the label 'lang/hi' with your PR. Thanks!

A small reminder for everyone, please use the label 'lang/hi' with your PR. Thanks!

@jayesh-srivastava Everyone can't edit labels on PRs. We maintainers have the rsponsibility to assign appropriate labels

@anubha-v-ardhan Oh okay! Got it.

Hey all, i would be starting to work on content/hi/containerization.md :)

Hey Everyone, I would be love to work on content/hi/container.md and content/hi/container-image.md

@kranurag7 Please only take one page at a time. Assigning content/hi/container.md to you

Sure @anubha-v-ardhan I will work on this. Thank you for assigning.

will work next on content/hi/_index.md

will work next on content/hi/_index.md

content/hi/_index.md is already localized with PR #300 as shown in the cheklist above. Please choose something else.

content/hi/api_gateway.md

Thats on me. Thanks. I will choose content/hi/api_gateway.md

Hi I would love to contribute to this issue.I would like to take  content/hi/client_server_architecture.md file.If anyone is not working on it please assign it to me.

@Aayush987 Thanks for coming up. The issue will be assigned to you if no one is working on it.

@kranurag7 Thanks

I wanted to ask if i have to do it via cloning on my system or i can simply do it in github on browser?

@Aayush987 You can do it in browser only.

Thanks @Aayush987. I've assigned the requested page to you. Please also consider joining #glossary-localization and @glossary-localization-hindi on CNCF Slack for discussions or conversations of any kind.

I wanted to ask if i have to do it via cloning on my system or i can simply do it in github on browser?

I's not sure if we can branch from dev-hi on GitHub itself. If yes, I see no harm in doing so. Note that the PR will have to target dev-hi and not main

Thanks @Aayush987. I've assigned the requested page to you. Please also consider joining #glossary-localization and #glossary-localization-hindi on CNCF Slack for discussions or conversations of any kind.

I wanted to ask if i have to do it via cloning on my system or i can simply do it in github on browser?

I's not sure if we can branch from dev-hi on GitHub itself. If yes, I see no harm in doing so. Note that the PR will have to target dev-hi and not main

Can you please send me CNCF slack link . I cant find it?

I would be working on content/hi/cloud_native_security.md.

My previous Pr on  content/hi/continuous_integration.md just needs approval , now i would like to work on  content/hi/canary_deployment.md.Plz assign it to me

Updated https://github.com/cncf/glossary/pull/417 to tracking issue

Hi, @Shubh28698 and @WebShivam!
Thank you for signing up for your respective pages.
We would like to get placeholder PRs for the same if you have begun work on the localization already.
Please also paste the tracking issue https://github.com/cncf/glossary/issues/270 in your PRs description, this helps us track.
Thank you for your help!

hi, taking content/hi/contributor-ladder/_index.md next.

@Garima-Negi Any updates on content/hi/contributor-ladder/_index.md. It's the last page required before we can launch.

hey, is there any issue I can work with?

hey, is there any issue I can help with?

Yeah, you can take a page from checklist in this issue's which is not already assigned to someone and localize it.

can I take up automation.md?

Hi! Can I work on localizing bare_metal_machine.md ?

can I take up transport layer security

Hi, I would like to contribute to content/hi/blue_green_deployment.md

is there anything i can help with? i would love to contribute in hindi glossary.

Hi, I would like to contribute to content/hi/service_mesh.md

Closing this issue as we don't have plans to use this anymore and we're doing the traditional make issue -> wait for triage -> make PR way


[ERROR] [Deployment] .env was not found. Check Contributing.md for details.

Describe the bug
I'm having issue with the deployment on vercel
To Reproduce
Steps I followed for deployment:

Sign in to Vercel or create a new account at https://vercel.com/
Clone this repository with git clone https://github.com/DenverCoder1/github-readme-streak-stats.git
You may also fork the repository and clone your fork instead if you intend to make changes
Enter the directory with cd github-readme-streak-stats
Switch branches to the vercel branch with git checkout vercel
Make sure you have the Vercel CLI installed
Run vercel and follow the prompts to link your Vercel account and select a project name
The app will be deployed to .vercel.app
Visit this link to create a new Personal Access Token (no scopes required)
Scroll to the bottom and click "Generate token"
Visit the Vercel dashboard and select your project, then click "Settings", then "Environment Variables".
Add a new variable with the key TOKEN and the value as your token from step 9 and click "Save".

Screenshots
Having the following .env related error.

@shariqmalik was getting this issue as well. What worked for me just  rename .env.example to .env , then add in your KEY token in file and save changes. Then vercel should update .env file if linked with repository

Steps 11 and 12 add the token to the app. After you do those steps it should work

@shariqmalik was getting this issue as well. What worked for me just rename .env.example to .env , then add in your KEY token in file and save changes. Then vercel should update .env file if linked with repository
FYI make sure your repo is private so token would not be exposed to public

I've created a .env file with the token and then runned vercel but it still doesn't work

@shariqmalik was getting this issue as well. What worked for me just rename .env.example to .env , then add in your KEY token in file and save changes. Then vercel should update .env file if linked with repository
FYI make sure your repo is private so token would not be exposed to public

I've created a .env file with the token and then runned vercel but it still doesn't work

I have even added the token to the app

I looks like you issue is, you are in the repo main branch and not the branch name "vercel"

@shariqmalik was getting this issue as well. What worked for me just rename .env.example to .env , then add in your KEY token in file and save changes. Then vercel should update .env file if linked with repository
FYI make sure your repo is private so token would not be exposed to public

I've created a .env file with the token and then runned vercel but it still doesn't work

I have even added the token to the app

It looks like your issue is, you are in the repo main branch and not the branch name "vercel"

I'm in the vercel branch

I mean edit .env file in vercel branch in the repo that is connected .. it looked like from in the screenshot you did it in main branch. If still not working just make a private repo with vercel branch contents, connect repo in vercel and then rename file(.env.example) and add key token and save

I mean edit .env file in vercel branch in the repo that is connected .. it looked like from in the screenshot you did it in main branch. If still not working just make a private repo with vercel branch contents, connect repo in vercel and then rename file(.env.example) and add key token and save

I've just created a new private repo, added the .env with the token in the vercel branch, recreated the project on verce, changed the Production Branch in vercel, added the token even in the vercel panel, but still don't working...

I've alerady turned off "Include source files outside of the Root Directory in the Build Step". Now i try to recreate the repo with the only vercel content as main branch but i think that nothing will change...

@trinib I tried but just like @Klay4 still having the same issue.

Let me know if there is anything I'm still missing.

@Klay4
Things worked for me.

Cloned or mirror this repository into a new private repository.
Import private repo to vercel.
Added TOKEN in Environment Variables, unchecked Include resource files outside of Root Directory..., and Hit deploy.
After deployment it won't work as the branch is main. To fix it go back to Settings -> Git and change the branch name from main to vercel.
To change the production branch, you must now push a commit.
I just added TOKEN to the .env file and commit it. I have no idea why I did this as I already added TOKEN to the environment variables.
The Production branch changed to vercel and it's working now.

Thanks a lot, @trinib & @DenverCoder1

I have added to the steps to redeploy the app after setting the environment variable as it seems to be required (https://github.com/DenverCoder1/github-readme-streak-stats/commit/9b0767eada70dd47180e41afe6205eb60755ac08).
Also, I'd recommend against putting active tokens in a GitHub repo even if it's private. If it's private it's less likely to be an issue, but files in GitHub repos are not a secure way of storing secrets. .env should only be used if running locally or on a dedicated server. If using a service, such as Heroku or Vercel, that provides other methods of inputting secrets, the service's recommended method should be used.


paml: 4.9i -> 4.9j

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/paml/versions.

WARNING: Package has meta.broken=true; Please manually test this package update and remove the broken attribute.

meta.description for paml is: '"Phylogenetic Analysis by Maximum Likelihood (PAML)"'.
meta.homepage for paml is: '"http://abacus.gene.ucl.ac.uk/software/paml.html"

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 4.9j with grep in /nix/store/3ljiz0nax6zgqwwxv2zc10vba4hzddzp-paml-4.9j

directory tree listing: https://gist.github.com/5c0e6b0672cd1d925b205a5da2cf19c2

du listing: https://gist.github.com/a0e59b189d6433e940940e9d313165cd

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)
1 package rebuild(s)
1 x86_64-linux rebuild(s)
0 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
0 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
paml

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/3ljiz0nax6zgqwwxv2zc10vba4hzddzp-paml-4.9j \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A paml https://github.com/r-ryantm/nixpkgs/archive/7c80fe2312ad8fa3a8d35bdc7f4d51f2062b8453.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/3ljiz0nax6zgqwwxv2zc10vba4hzddzp-paml-4.9j
ls -la /nix/store/3ljiz0nax6zgqwwxv2zc10vba4hzddzp-paml-4.9j/bin

pushed 605e88517b16bdfee891bc90c6dc2653e7eb0ec5


Use APP_ENV for Sentry environment

:clipboard: #771
:clipboard: https://github.com/omisego/devops/pull/103
Overview
This PR is dependent on https://github.com/omisego/devops/pull/103. This leverages APP_ENV to pick up our Sentry environment instead of MIX_ENV.
Changes
Describe your changes and implementation choices. More details make PRs easier to review.

Use APP_ENV instead of MIX_ENV: 1a3e70a7
Add additional Sentry tags like eth_network, mix_env, and eth_node
Remove source code context feature until a later PR. Would make our deploys a bit more difficult at the moment.

Testing
ETH_NODE="bar" ETHEREUM_NETWORK="foo" APP_ENV="prod" MIX_ENV=dev mix sentry.send_test_event

Coverage remained the same at 79.358% when pulling c49a1c88b50e87b6d09f2f954f961935be673a0f on achiurizo/tags-for-sentry into 7a66b808ae827c13eb166135b2ed47d91af8b972 on master.


Fixes to Buffy and first commit for Deliria character sheet.

Changes / Comments
Roll20 Requests
Include the name of the sheet you made changes to in the title.

If changes fix a bug or resolves a feature request, be sure to link to that issue.
For pull request that change multiple sheets please confirm these changes are intentional for all sheets. This will help us catch unintended submissions.
When updating existing sheets if you are changing attribute names please note what steps you have taken, if any, to assist players in keeping their data.

What's the point of adding a new license, when everything in this repository is already under MIT license?

Didn't realise that. The license I've selected is a copyleft license, effectively meaning people are welcome to use it, provided its attributed, but all uses must be in the public domain.

Greetings @Kirintale
Everything accepted into this repository must operate under the MIT license. I can appreciate the desire to use the GNU license but we currently do not have the support or process in place to allow for a variety of individual licenses. To accept the sheet into the repository the license will need to be removed.
Thank you,
Cassie

Understandable. I'll fix it tomorrow. No rush for getting this commit in.

Hi If you can cancel this. I can do the update later and try to commit again during the week.
Ah so I can use table for tabular data but not for layout. Strange, but I'll do an update later.
Have a wonderful day.

Tables for formatting is harder to change and maintain and change, I've witnessed it first hand with one sheet I took over as maintainer.
A few more examples and reasons.

@Anduh
Thank you for your input and the links. I'll do the updates later.


Rename spack-install to spack-setup

This Change

Renames spack-install to spack-setup. This is to help avoid confusion since spack install has the connotation of using spack to install other software and will generally be executed by the spack-execute module.
Updates all references to spack-install.
This also updates documentation for the spack-setup module to demonstrate proper usage and limit the scope to not include installation of compilers and packages.

It may help to review the commits independently. The first commit is a no-op other than renaming.
Submission Checklist
Please take the following actions before submitting this pull request.

Fork your PR branch from the Toolkit "develop" branch (not main)
Test all changes with pre-commit in a local branch #
Confirm that "make tests" passes all tests
Add or modify unit tests to cover code changes
Ensure that unit test coverage remains above 80%
Update all applicable documentation
Follow Cloud HPC Toolkit Contribution guidelines #

I will send a follow on pr that will hand migration.


Performance in few-shot setup

Awesome work @cyrilou242, thanks!
I wonder if anyone tested the ftcc method against gzip in the few-shot setup, k={5, 25,...}, I thought this was like the highlight of the gzip-paper, showing that gzip can perform reasonable with small training data. I guess if training data is sufficient, there's hardly a way to beat BERT with any non-neural method.

the ftcc method against gzip in the few-shot setup, k={5, 25,...}, I thought this was like the highlight of the gzip-paper

I have not tested yet. I think the result is artificial and not that interesting.

it's not clear to me how this setup is realistic and corresponds to a real problem.
the accuracy results are bad anyway and would not be usable in real-life
when I looked at the code, there was no random seed (maybe this has changed) to reproduce with the same samples.
of course statistical methods would beat machine learning with the given few shot learning. Bag of words counts would most likely be easier to understand.
the datasets had so many train/test leakage issues

I can have a try though. Maybe this week-end.

Thanks!

I have not tested yet. I think the result is artificial and not that interesting.

Why do you think it is not interesting? I'd even say that the underlying KNN doesn't really make sense if there is much training data (it is too costly). Even if accuracy results may be low, it all depends if it is better than baselines.
Maybe it's of interest to you, I ran some experiments with bag-of-words and gzip on some data sets, with different number of shots. Sometimes the accuracy results are not even that bad (like on DBPedia). These data sets also don't have the issues of data contamination (afaik).

Image is from this write-up.
Again, ftcc is very cool and interesting work!

Why do you think it is not interesting?

I'm an industry guy, I may be biased.
Contrary to some cases for which labelled data is hard to obtain, labelled text classification data is pretty easy to obtain.
So let's say my goal is to have a good model with a given budget. I start with a dataset of 5 observations.
I have two choices: spend money on improving the model or spend money on improving the data.
It is pretty obvious at 5 observations that any dataset improvements will improve the system order of magnitudes more than improving modelling.
Also, a dataset improvement is often a better investment than a model improvement: labelled data tends to live longer than a particular model.
I can see a use case for cold-start in labelling UI like https://prodi.gy/ though.
Anyway given you have a nice paper with nice numbers, I can give it a try.
Do you have the code with a random seed or an exact list of the samples taken for the few-shots?

It is pretty obvious at 5 observations that any dataset improvements will improve the system order of magnitudes more than improving modelling.

That's true, now I get where you're coming from!
But, I mean, doesn't this then beg the question, what is even the purpose of using a KNN for text classification (regardless of which distance function you take)?

Do you have the code with a random seed or an exact list of the samples taken for the few-shots?

Unfortunately not, my code is just changing the distance function in the gzip paper (and fixing the evaluation).

btw, I now tested a SVM+tfidf bag of words approach. It performs very strongly on full training data (almost similar level as BERT), but is also strong in few-shot setup:
Here's the preliminary table (I'm also gonna update my note with it, soon):

(SIMPLE in the image above has become BowDist)
So I am starting to  wondering a bit why was the gzip paper hyped so much in the first place.

I updated my paper with my more results now.
Seems as if BoW approaches are very strong. When used as "untrained" distance measure in KNN it performs better than GZIP distance in KNN, when used inside a trained classifier, it almost achieves same performance as BERT.
It'd still be interesting for me how ftcc relates to all of this, and also whether it also works with fewer training data, but I'm gonna close this issue for now.


Syncing stalls for a couple of hundred seconds

We have the problem that syncing stalls now and then for a couple of hundred of seconds.
Example (~400s):
handle_ledger_update{milestone_index=4855283 created=7 consumed=5}: inx_chronicle::inx: close time.busy=13.8ms time.idle=374ms
handle_ledger_update{milestone_index=4855284 created=8 consumed=8}: inx_chronicle::inx: close time.busy=13.6ms time.idle=292ms
handle_ledger_update{milestone_index=4855285 created=4 consumed=5}: inx_chronicle::inx: close time.busy=15.7ms time.idle=149s
tower_http::trace::on_failure: response failed classification=Status code: 503 Service Unavailable latency=23239 ms
handle_ledger_update{milestone_index=4855286 created=4 consumed=3}: inx_chronicle::inx: close time.busy=13.1ms time.idle=273s
tower_http::trace::on_failure: response failed classification=Status code: 503 Service Unavailable latency=19715 ms
handle_ledger_update{milestone_index=4855287 created=7 consumed=5}: inx_chronicle::inx: close time.busy=12.2ms time.idle=34.8s
handle_ledger_update{milestone_index=4855288 created=7 consumed=6}: inx_chronicle::inx: close time.busy=16.9ms time.idle=433ms
handle_ledger_update{milestone_index=4855289 created=4 consumed=5}: inx_chronicle::inx: close time.busy=12.6ms time.idle=389ms

We think, this could be connected to the analytics endpoint /ledger/richest-addresses that seems to be quite heavy on the database:

(at the time the screenshot was taken, the chronicle-service was not available on the load-balancer and there were about
20 such queries running).
This is the view of the load-balancer (3.00 members is when all chronicles healthy):

CPU-Profile looks about this on the Primary Mongo:

We think, that changing the implementation from range-queries to incremental could solve the problem because it would lower the load significantly.
Naively thought it would be like this:

run the query once at startup
only update the top100 richest address in memory on each new milestone

Probably token-distribution analytics is similar (but I don't know).
Chronicle version
Chronicle rc.1

I tried caching the heavy endpoints for an hour but it seems it's only a temporary measure:

The spikes are exactly 1h apart and situation seems to get slowly worse over time :thinking:

This is fixed in 2.0 using new collections to track the latest address analytics.


bug(datepicker): Months in Estonian language are too long

Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description
If language is set to Estonian ('et') the month names in datepicker are too long (in other languages like English they're cut of at 3 characters)

Reproduction
Stackblitz: https://stackblitz.com/edit/components-issue-mrozgp?file=src/app/example-component.ts
Steps to reproduce:

Set DateAdapter locale to Estonian ('et')
Open datepicker
Click in the month/year title
Select year
Months view should be displayed with too long names

Expected Behavior
Months should be cut off at 3 characters as in other languages
Actual Behavior
Month names are displayed enterely
Environment

Angular: 12.2.9
CDK/Material: 12.2.9
Browser(s): Chrome 95.0.4638.69 (Official Build) (x86_64)
Operating System (e.g. Windows, macOS, Ubuntu): MacOS Monterey (12.0.1)

@llaz00 This looks like the behavior of the browser's Intl API:
var f = Intl.DateTimeFormat('et', {month: 'short'});
f.format(new Date('Sept 1, 2021'));

// returns 'september'

You have a couple options here:

Use an alternate DateAdapter implementation that has different formatting for Estonian
Add a custom DateAdapter and manually control the formatting


adding click open functionality for cxtmenu

This allows a click-open functionality similar to the neo4j-browser

Does this work in conjunction with other events, e.g. can I specify click cxttapstart?  If it breaks swiping (the main usecase for the extension), it could be a problem.

This currently only changes behavior if click is defined as the only openMenuEvent - so it should not cause issues with swiping interactions

@kamranjon It seems to work OK for me even with openMenuEvents: 'cxttapstart taphold click'.  I noticed some things though:

click should probably be tap to be device agnostic.
click allows you to click the background or another element when doing the second click to select the menu item.  This means that it could bring up the menu for another node or bring up some other UI in an app.  It would be good if this could be prevented somehow.

@maxkfranz thanks for the feedback - I thought about this for tap - I will implement that - and for the second point I will look into preventing the event. Thanks.


Fix title if creating table from query

Fixes #4801
Actually using the table.duplicate that internally duplicate table from the query, this changes the title depending on that state
@javierarce for code review

:+1:

Frontend tests were OK :+1: (details)


Support using a function instead of a plain string for urlText

I'd like to expand the syntax used for markdown in the urlText, like by specifying the height & width (an extension to normal markdown):
![file_name)(/imgs/123456789.png =120x60)

Though I don't want to hard-code a default size, and instead customize it based on data returned from the server. My thought is that a function could be set for urlText that will be passed in the data returned from the server. The function is expected to return a string that will be placed literally into the textarea (so if you opt to use the function, you must make sure to include the attachment's name in the string).

I'll add another hook for it, i think it can be added to the onFileUploadResponse response and add another parameter in which the resulting text can be modified that will be inserted in the textarea

Here's one way it can be done: https://github.com/Rovak/InlineAttachment/pull/70