Cloud Foundry Deployment changes

Under "Deploying to Cloud Foundry" section in the readme, it's written, "Please adjust to your particular Cloud Foundry deployment environment."
Can anyone please tell me where and what changes do I have to make according to CF environment?
I have deployed all the abacus application, after that, I deployed "abacus-cf-applications" and I have updated the manifest file with AUTH_SERVER, CF_CLIENT_ID and CF_CLIENT_SECRET but in the logs the cf-application, I am getting following errror

e-abacus-paging 55 Missing CF token
2017-09-19T14:55:34.24-0700 [App/0] OUT 2017-09-19T21:55:34.249Z e-abacus-oauth 55 Error getting OAuth bearer access token, { message: 'Unable to get token endpoint from oauth server' } - Error: Unable to get token endpoint from oauth server

Hi @Nishanksingla,
In app bridge we use the following CF environment:
### CF installation ###
API: https://api.<domain>:443
AUTH_SERVER: https://api.<domain>:443
JWTALGO: RS256
JWTKEY: -----BEGIN PUBLIC KEY-----
<  UAA public key >
-----END PUBLIC KEY-----

PRIMARY_DOMAIN: <domain>

### Abacus landscape ###
CF_CLIENT_ID: abacus-cf-bridge
CF_CLIENT_SECRET: <secret>
CLIENT_ID: abacus-linux-container
CLIENT_SECRET: <secret>
COLLECTOR: abacus-usage-collector
EUREKA: abacus-eureka-plugin
SECURED: true

### Abacus specific ###
EVAL_VMTYPE: vm
DEBUG: e-abacus-*
NODE_MODULES_CACHE: false
NODE_OPTS: --max_old_space_size=256
BATCH_SIZE: 50
THROTTLE: 100

### DB ###
DBALIAS: db-common
DBCLIENT: abacus-mongoclient

I separated the variables in several groups:

CF specific
can be obtained from your deployment manifest or the CF CLI target.
Abacus landscape
includes installation specific variables: UAA client ids and secrets and application names (if you ever need to changed them).
Abacus specific
related to scaling, debugging, plan handling
DB
here you can specify what DB to be used (service or external URL), type of database and finally specify if you need app-level (poor-man) sharding if needed.

The accumulators use the following set of variables:
ACCOUNT: abacus-account-plugin
AGGREGATOR: abacus-usage-aggregator
API: https://api.<domain>:443
AUTH_SERVER: https://api.<domain>:443
BATCH_SIZE: 50
CLIENT_ID: abacus
CLIENT_SECRET: s3cr3tpass?system!
CONF: large
DBALIAS: db-common
DBCLIENT: abacus-mongoclient
DEBUG: e-abacus-*
EUREKA: abacus-eureka-plugin
EVAL_TIMEOUT: 800
EVAL_VMTYPE: vm
JWTALGO: RS256
JWTKEY: -----BEGIN PUBLIC KEY-----
<UAA public key>
-----END PUBLIC KEY-----

NODE_MODULES_CACHE: false
NODE_OPTS: --max_old_space_size=256
PRIMARY_DOMAIN: <domain>
PROVISIONING: abacus-provisioning-plugin
REPLAY: 1800000
SAMPLING: 1h
SECURED: true
SLACK: 5D
THROTTLE: 100

Note that we have SAMPLING above, that means accumulator will create a new accumulated data document every hour. This reduces the amount of data in the DB, but also means we can lose 1h of data in case of crash, corrupted database, etc.
Also we do a replay of our input DB to guarantee no data is stuck in our input DBs by supplying REPLAY with a value of 30 minutes. This means that we allow our apps to be down for updates for up to 30 minutes.

Hi @hsiliev
Thank you for the information. I have few doubts.
I am writing the values in the angle brackets which I have understood, please correct me If I am wrong.
In CF Installation section:
API: < CF API Link( https://api.system.....) >
AUTH_SERVER: < UAA server link( https://uaa.system...) >
In Abacus landscape
CF_CLIENT_ID:
CF_CLIENT_SECRET:
CLIENT_ID:
CLIENT_SECRET:
What is Client_ID and Client_Secret and from where to get values for these fields?
Also, these configurations are for cf app bridge, what changes do I have to make in abacus applications for cloud foundry. Please help me with that also.

There are several UAA clients Abacus needs to function correctly in secured mode. More info on security concept doc: https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/doc/security.md
All of these are created by our Concourse pipeline: https://github.com/cloudfoundry-incubator/cf-abacus/tree/master/etc/concourse
Additionally each Resource Provider needs to authenticate via token. We automate this via Abacus broker project: https://github.com/cloudfoundry-incubator/cf-abacus-broker

Thank you for the information.
After looking at all the code and documents, I am stuck at one thing.
As CouchDB bosh release is not out yet, how can I run the pipeline without the database ?
And if I do create-db-service field as false in deploy-pipeline-vars.yml, will the pipeline deployment use abacus-pouchserver application for storing everything?

You can:

use user-provided services
specify DB URL directly with DB environment variable in manifest

Setting create-db-service: false will help you do each of the above, as you most probably won't need a DB service to be created in this case.

Ok.
For the demo purpose, how can I use abacus-authserver-plugin with cf-app-bridge application?
How to create client_id and client_secret for cf-app-bridge application in abacus-authserver-plugin?
Also, How to point to abacus-pouchserver in cf-app-bridge manifest.yml?
Initially, I deployed all the abacus application and app bridge application. I was getting the following error.
Error getting OAuth bearer access token, { message: 'Unable to get token endpoint from oauth server' } - Error: Unable to get token endpoint from oauth server
I would appreciate a lot if you can help.

app-bridge cannot work without CF Cloud Controller (CC) as it pulls events with admin read-only scope.
You could run app-bridge with authserver-plugin, but you still need to connect app-bridge to CF CC. And the token validation in CC should not allow you to use tokens issued by authserver-plugin instead of UAA to fetch the app-usage events.
So I would recommend to use UAA instead.
Have in mind that pouchserver is used for testing and developing Abacus. You should be able to use it by simply deploying it to CF and using DB: abacus-pouchserver in app manifests. If you are using HTTPS you might need to specify the full path: DB: https://abacus-pouchserver.<domain>:443

Ok.
What I have done is that for app-bridge application I have register abacus-cf-applications uaa client to talk to CC and I have entered the following
    CF_CLIENT_ID: abacus-cf-applications
    CF_CLIENT_SECRET: < my genereated secret >

For resource provider authorization using authserver-plugin I have entered the following configuration which comes as default as I don't know how to create client_id and client_secret with authserver-plugin. I hope both values are right.
 CLIENT_ID: abacus-linux-container
 CLIENT_SECRET: secret

Please let me know if this is right.

Seems in synch with the config we use (above)

My only question is
So the client_secret value is "secret"?
Client_ID = "abacus-linux-container" and Client_Secret= "secret" is already added in authserver-plugin or do I have to add somehow ?

On Sep 21, 2017, at 12:12 PM, Hristo Iliev<EMAIL_ADDRESS>wrote:
Seems in synch with the config we use (above)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

The pipeline (etc/concourse) uses the following script to create UAA users:
https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/etc/concourse/scripts/create-uaa-clients#L42-L49
The default client names we use are:
SYSTEM_CLIENT_ID=abacus
BRIDGE_CLIENT_ID=abacus-cf-applications
CONTAINER_CLIENT_ID=abacus-linux-container
HYSTRIX_CLIENT_ID=abacus-hystrix
The client names and secrets can be changed. What's important is that they contain the right scopes, that we use for authentication/authorization.

I am not using UAA for authorization now just the abacus-authserver-plugin . I have created CF_Client_Id and CF_Client_Secret in UAA that can read cloud controller API's.  Rest of the authorization and authentication stuff I am trying to do with abacus-authserver-plugin .
That's why I want to know the values of Client_ID and Client_Secret for abacus-authserver-plugin as I am not creating UAA client right now.

Arbitrary values would do. As I said we care for the scope. Client ID and Secret are used for authentication and Auth Server does not support any as it is a dummy implementation as all Abacus plugins.

@Nishanksingla Any progress or new issues with the setup?

Hey @hsiliev
Thanks for checking in. Unfortunately, no progress.
This what I am getting again and again.

Error getting OAuth bearer access token, { message: 'Unable to get token endpoint from OAuth server' } - Error: Unable to get token endpoint from OAuth server

In cf app bridge application as well as abacus-usage-collector.
In cf app bridge application, I am also getting Missing resource provider token
I don't what's going wrong, I have created UAA clients for the resource provider, system, and CF_Client.
Following is my manifest file for Cf app bridge application

applications:

name: abacus-cf-applications
host: abacus-cf-applications
path: .cfpack/app.zip
instances: 1
memory: 512M
disk_quota: 512M
env:
CONF: default
DEBUG: abacus-cf-, e-abacus-
COLLECTOR: https://abacus-usage-collector.run.aws-*****
EUREKA: https://abacus-eureka-plugin.run.aws-*****
API: https://api.system.aws-*****
DB: https://abacus-pouchserver.run.aws-*****
DBCLIENT: abacus-couchclient
CF_CLIENT_ID: abacus-cf-applications
CF_CLIENT_SECRET: < SECRET >
NODE_MODULES_CACHE: false
SECURED: true
AUTH_SERVER: https://uaa.system.aws-*****
CLIENT_ID: abacus-linux-container
CLIENT_SECRET: < SECRET >
JWTKEY: |+
-----BEGIN PUBLIC KEY-----
public key
-----END PUBLIC KEY-----
JWTALGO: RS256

I am not getting what oauth server is doing. Also, cf-app-bridge or any other application is even talking to UAA or not?

Can you post the scopes of your clients? I want to check that they are correct. You can also see if oauth is working by adding abacus-oauth* to DEBUG.
Due to #627 you have to specify urls like this https://<host>:443
Not sure if this is paste & formatting problem, but DEBUG should have wildcards at the end of the strings: e-abacus* for example

Below are the commands that I used to Uaac Clients:
Resource provider

uaac client add abacus-linux-container --name abacus-linux-container --authorized_grant_types client_credentials --authorities abacus.usage.linux-container.write,abacus.usage.linux-container.read --scope abacus.usage.linux-container.write,abacus.usage.linux-container.read --secret < SECRET >

System

uaac client add abacus --name abacus --authorized_grant_types client_credentials --authorities abacus.usage.write,abacus.usage.read --scope abacus.usage.write,abacus.usage.read

CF_Client to take to cloud controller api

uaac client add abacus-cf-applications --name abacus-cf-applications --authorized_grant_types client_credentials --authorities cloud_controller.admin

Yeah, that was the formatting issue. I will add 443 and will try again but the urls point to 443 ports only.

I made it work. The problem was in naming convention.
In our case AUTH_SERVER has its own endpoint which starts with https://uaa.system... but when I looked at the code, the code makes a "GET" a call at the CF API endpoint(which is https://api.system.../v2/info) and fetch the auth server endpoint which again is https://uaa.system...

@Nishanksingla Good point. Created a new issue to remove the API endpoint.
I'll close this issue, but if you have more questions feel free to reopen it.

Hi @hsiliev
Thank you for all the help. I appreciate it.
Now the authentication and authorization stuff is working, I am getting following error

The usage submitted is older than 600000 milliseconds

After the changing the slack_window to 5d still, I am getting the same error.

The usage submitted is older than 432000000 milliseconds.

I don't think I understand the slack_window completely. According to what I understood, Slack_window is the time difference between last usage document and current usage document.
But each usage document has 2 times, start and end.
Please correct me if I am wrong.
I also ran the abacus-get-usage.sh  script for many org_id's but always I am getting following empty response

{
"id": "k/001fd8e7-0162-4eeb-84d9-30d1f4110c9c/t/0001506636149539",
"organization_id": "001fd8e7-0162-4eeb-84d9-30d1f4110c9c",
"start":<PHONE_NUMBER>000,
"end":<PHONE_NUMBER>539,
"resources": [],
"spaces": [],
"windows": []
}

Slack is used to allow Resource Providers to send usage from previous month. They might be slow (or unable to send) due to crashes, network partitioning, updated.
A workaround would be to extend this window to day 31d. This will allow you to fetch some events (but probably not all).
The solution is to start a billing epoch for CF:
https://github.com/cloudfoundry-incubator/cf-abacus/wiki/Installation#billing-epoch
This will reset all events to the current time, so bridges can transfer them. It also makes them fit to the slack window.

Hi @hsiliev,
Thank you for the help. I am able to solve the issue and run the abacus.
Now I have to usage summary reports.
The only problem I am facing now that How to read usage summary report? and How to make sense of it?.
There are so many fields that it is getting hard to understand it. I tried to find the documentation but couldn't find anything. Also, I read the usage summary report API but was not able to find the answers.
I would appreciate a lot if you can help me regarding this.

The report contains 3 different roots:

the organization as a whole
breakdown for different spaces in the org
breakdown for resources.

You can drill-down based on what your UI or integration layer is interested in. If you don't need all the data I guess you can go with just aggregated usage per resource. This should give you monthly usage for linux-container, object-store, PostgreSQL or whatever resource providers submit data in Abacus.
The report works with time windows structures documented here:
https://github.com/cloudfoundry-incubator/cf-abacus/wiki/Time-Windows
To minimize the report size you can use GraphQL:
https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/doc/api.md#graphql-usage-query

I understand the format of the report but I have few questions.
1.) Below is the sample window which I got in a report of an organization

"windows": [
[
null
],
[
null
],
[
null
],
[
{
"charge": 0.02621032444722222
},
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null
],
[
{
"charge": 0.02621032444722222
},
null,
null
]
]

The window is Array of 5 Arrays in which 4th array is Minute's and the 5th array is Seconds according to https://github.com/cloudfoundry-incubator/cf-abacus/wiki/Time-Windows. In my case minute array is always of length 32 and seconds array is of length 3.
I don't understand these lengths and what's the data inside each array represents.
2.) When I am fetching a usage summary report of an org with the org_id, I am getting the report from which date to which date? Start and end is a very big number.
3.) Why is 'consumed' a big negative number? If I look inside resources->plans->aggregated_usage>windows, I see the following object which I am not getting.

"quantity": {
"consuming": 64.75,
"consumed": -4112402000
},
"cost": {
"price": 0.00028,
"consuming": 64.75,
"consumed": -4112402000
},
"summary": 93.60830159722222,
"charge": 0.02621032444722222
},

The report is very confusing to get.

Actually your daily window is from 32 days. I guess you set SLACK to 32 days, since there is 1:1 connection between the two. Abacus has to store 32 days back in time to allow usage to be submitted 32 days back in time.
The report is always from a month: from start of the month to the date you provided in the request. If no date is provided, the current month (and end-time = now) is used.
consumed is an internal state, keeping the "memory balance". The sign is used to keep track if the memory is used or unused (app started or stopped). In short: you should not use its value.

I have set the SLACK to 31d as you suggested. What is the order of five arrays in windows array? Is it seconds, minute, hours, days and months  or  months, days, hours, minute and seconds.
According to what you are saying it's the first order but according to the [https://github.com/cloudfoundry-incubator/cf-abacus/wiki/Time-Windows] (https://github.com/cloudfoundry-incubator/cf-abacus/wiki/Time-Windows) it's the second order.
Please correct me.
So if I remove the SLACK from the manifest file, I won't see 32 length array. Let me give it a try.

Hi,
The correct order is: seconds, minute, hours, days and months. The example on the wiki is a conceptual representation as JSON and is not the real structure. I will edit it to avoid confusion in the future.
Best regards,
Hristo

Ok Thanks.
So I changed SLACK to 5D and following is the windows that I got. It seems that the day array is always one plus the number of slack days. I don't know why and don't know the meaning of each entry.

"windows": [
[
null
],
[
null
],
[
null
],
[
null,
null,
null,
null,
null,
{
"quantity": {
"consuming": 43.5625,
"consumed": -2766741500
},
"cost": {
"price": 0.00028,
"consuming": 43.5625,
"consumed": -2766741500
},
"summary":<PHONE_NUMBER>2222224,
"charge": 0.038774497222222225
}
],
[
null,
{
"quantity": {
"consuming": 43.5625,
"consumed": -104389341500
},
"cost": {
"price": 0.00028,
"consuming": 43.5625,
"consumed": -104389341500
},
"summary": 1183.9803472222222,
"charge": 0.3315144972222222
}
]
]

But still, I don't get the meaning of each entry in each array. Like what does the 6th entry of day array means?

Hi,
I am preparing a wiki page that would explain the time windows better. However it is still under development and I will just paste here a part of it:
Time windows
Usage accumulation happens in the so called time windows. These are predefined timeframes that span back from the usage  processing time (the time when the usage hit Abacus). The current periods are [second, minute, hour, day, month]. Each usage consumption is accumulated to a time window if its end time falls in the same time interval.
Example:
Accumaltor recieves usage (number_of_api_calls: 2) with the current timestamp as end time. There is a previous usage (number_of_api_calls: 1) reported in the last minute and the current accumulated document time windows are set as: [1, 1, 1, 1, 1] (s, m, h, D, M). The new usage will accumulate accross all time windows (except second) as its end time is right after (less than a minute) the last sent usage consumption. So the resulting accumulated document will look like this: [1, 3, 3, 3, 3].
Example 2:
Accumaltor recieves usage (number_of_api_calls: 2) with the current timestamp as end time. There is a previous usage (number_of_api_calls: 1) reported 5 hours ago and the current accumulated document time windows are set as: [1, 1, 1, 1, 1] (s, m, h, D, M). The new usage will accumulate to the day and month time windows but not with the minute and hours windows. So the resulting accumulated document will look like this: [1, 1, 1, 3, 3].
Slack period
Abacus checks every usage consumption document date and if it is too old the document gets rejected. The period of time in the past for which usage documents are accepted is configurable and is called slack period. By default the slack period is configured to 10m (10 minutes). It can be set by setting the env variable SLACK.
The slack period has direct relation to the time windows. The time dimension for which the slack period is defined has multiple values for every accumulated metric. The number of accumulated values is the same as the slack period plus one (the current value). This way when a new usage is received and the consumption end date is in the past (but still not older than the slack period) it will not be accumulated in the current entry but to the corresponding entry inside the time window dimension.
Example:
Lets have a slack period configured for the last 3 days:
export SLACK = 3D

The time windows in the accumulator document (metric number_of_api_calls) will loook like this (s, m, h, D, M):
[ [null], [null], [null], [null, null, null, null], [null] ]

Because the slack period is set to 3 days the resulting time window for the day accumulation will have 3 entries (current + 3 days).
Lets assume we have a consumption reported (number_of_api_calls: 1). The result will look like this:
[ [null], [ { 'current': 1 } ], [ { 'current': 1 } ], [ { 'current': 1 } , null, null, null], [ { 'current': 1 } ] ]

The current day time window contais the value of 1.
If there is another usage reported (number_of_api_calls: 2) after 24 h (1 day) the resulting doc is update as follows:
[ [null], [ { 'current': 2 } ], [ { 'current': 2 } ], [ { 'current': 2 } , { 'current': 1 }, null, null], [ { 'current': 3, 'current': 1 } ] ]

The minute and hour values equal the new usage as in the last 60 min there were no other usages. The day usage is also 2 as for the last 24 h we don't have other usages. However as we have a configured slack period of 3 days the value for the previous day is not lost but shifted in the array with one index. The month value accumulates the new value as they are both in the same timeframe.
Lets now receive another usage (number_of_api_calls: 3) but its end date is from the previous day. Than we get:
[ [null], [ { 'current': 2 } ], [ { 'current': 2 } ], [ { 'current': 2 } , { 'current': 4, 'previous': 1 }, null, null], [ { 'current': 6, 'previous': 3 } ] ]

The new usage is not accumulated in the minute or hour time windows as it is old. It is not part of the current daily accumulation but must be accumulated to the value of the previous day. The accumulated value for the month is updated too.


unit tests (test_plot) failing with python 3.7

Using python 3.7.7 I get the following errors when running pytest:
E       ValueError: unsupported pickle protocol: 5
This is solved by moving to python 3.8 but the minimal version of python should be documented somewhere (or maybe an older pickling protocol can be used for the tests?)

This is likely due to the predicu integration, you can ignore the failing test, it's not currently used in the running servers in any way.
We shouldn't use pickles indeed as they are not portable across python version (cc @tgy ), there is still active development happening for predicu (e.g. https://github.com/icubam/icubam/pull/240) and we'll try to address this.
BTW, if we want to support Python 3.7 we should add it to CI.

Interestingly the issue happens when loading HDF5 file,
    @pytest.mark.parametrize("name", PLOTS)
    def test_generate_plots(name, tmpdir, monkeypatch):
      output_dir = str(tmpdir.mkdir("sub"))
      test_public_data_path = os.path.join(BASE_PATH, "tests/data/public_data.h5")
>     test_public_data = pd.read_hdf(test_public_data_path, 'values')
[...]
    def fromarray(self, array):
        # We have to check for an empty array because of a possible
        # bug in HDF5 which makes it claim that a dataset has one
        # record when in fact it is empty.
        if array.size == 0:
            return None
>       return pickle.loads(array.tostring())
E       ValueError: unsupported pickle protocol: 5

so this will be resolved once https://github.com/icubam/icubam/pull/240 is merged, as it removes the use of these files.

@rth HDF uses pickle to store some objects (here datetime objects I think) because it does not have native support for them... anyhow, like you said this is not an issue anymore.

Should be resolved in https://github.com/icubam/icubam/pull/240


Binding Issues on Linux

Still unable to bind properly on Linux 5.1.15-arch1-1-ARCH
bundle exec falcon serve produces
0.08s    error: <Async::Task:0x2b1ccafd6a1c failed> [pid=23552] [2019-06-27 13:56:32 +1200]
               |   Errno::EADDRINUSE: Address already in use - bind(2) for <IP_ADDRESS>:9292
               |   → /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224 in `bind'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224 in `block in bind'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:165 in `build'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:223 in `bind'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/address_endpoint.rb:43 in `bind'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:78 in `bind'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:192 in `bind'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:101 in `block in bound'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:203 in `block in each'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:93 in `block in each'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:84 in `block in each'
               |     /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227 in `each'
               |     /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227 in `foreach'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:83 in `each'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:92 in `each'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:202 in `each'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:100 in `bound'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/shared_endpoint.rb:29 in `bound'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/falcon-0.33.11/lib/falcon/command/serve.rb:109 in `block in run'
               |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-1.19.0/lib/async/task.rb:210 in `block in make_fiber'
bundler: failed to load command: falcon (/home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/bin/falcon)
Errno::EADDRINUSE: Address already in use - bind(2) for <IP_ADDRESS>:9292
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224:in `bind'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224:in `block in bind'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:165:in `build'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:223:in `bind'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/address_endpoint.rb:43:in `bind'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:78:in `bind'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:192:in `bind'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:101:in `block in bound'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:203:in `block in each'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:93:in `block in each'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:84:in `block in each'
  /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227:in `each'
  /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227:in `foreach'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:83:in `each'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:92:in `each'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:202:in `each'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:100:in `bound'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/shared_endpoint.rb:29:in `bound'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/falcon-0.33.11/lib/falcon/command/serve.rb:109:in `block in run'
  /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-1.19.0/lib/async/task.rb:210:in `block in make_fiber'

Then using a Procfile with web: bundle exec falcon serve gives a slightly different error in that it has a command not found line.
14:02:50 web.1  | started with pid 24248
14:02:50 web.1  |  0.08s    error: <Async::Task:0x2ab7918b70e0 failed> [pid=24248] [2019-06-27 14:02:50 +1200]
14:02:50 web.1  |                |   Errno::EADDRINUSE: Address already in use - bind(2) for <IP_ADDRESS>:9292
14:02:50 web.1  |                |   → /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224 in `bind'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224 in `block in bind'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:165 in `build'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:223 in `bind'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/address_endpoint.rb:43 in `bind'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:78 in `bind'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:192 in `bind'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:101 in `block in bound'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:203 in `block in each'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:93 in `block in each'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:84 in `block in each'
14:02:50 web.1  |                |     /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227 in `each'
14:02:50 web.1  |                |     /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227 in `foreach'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:83 in `each'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:92 in `each'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:202 in `each'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:100 in `bound'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/shared_endpoint.rb:29 in `bound'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/falcon-0.33.11/lib/falcon/command/serve.rb:109 in `block in run'
14:02:50 web.1  |                |     /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-1.19.0/lib/async/task.rb:210 in `block in make_fiber'
14:02:50 web.1  | bundler: failed to load command: falcon (/home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/bin/falcon)
14:02:50 web.1  | Errno::EADDRINUSE: Address already in use - bind(2) for <IP_ADDRESS>:9292
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224:in `bind'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:224:in `block in bind'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:165:in `build'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/socket.rb:223:in `bind'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/address_endpoint.rb:43:in `bind'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:78:in `bind'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:192:in `bind'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:101:in `block in bound'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:203:in `block in each'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:93:in `block in each'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:84:in `block in each'
14:02:50 web.1  |   /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227:in `each'
14:02:50 web.1  |   /home/simon/.rbenv/versions/2.5.1/lib/ruby/2.5.0/socket.rb:227:in `foreach'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/host_endpoint.rb:83:in `each'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/ssl_endpoint.rb:92:in `each'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-http-0.46.2/lib/async/http/endpoint.rb:202:in `each'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/endpoint.rb:100:in `bound'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-io-1.23.1/lib/async/io/shared_endpoint.rb:29:in `bound'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/falcon-0.33.11/lib/falcon/command/serve.rb:109:in `block in run'
14:02:50 web.1  |   /home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/async-1.19.0/lib/async/task.rb:210:in `block in make_fiber'
14:02:51 web.1  | exited with code 1
14:02:51 system | sending SIGTERM to all processes

Without falcon running lsof -I :9292 without falcon running?

Yup, no output, if something else was running on that port version 0.31.0 wouldn't be able to bind to that address either, but it can with the --reuse-port option passed.

@destructobeam thanks for the quick reply. I understand what you are saying but I'm having a hard time reconciling it with my understanding of what Falcon is doing.
The port is bound before forking:
https://github.com/socketry/falcon/blob/6d4ef205e2ef7a7e2ab84dbcd6fd2e6ec524c68f/lib/falcon/command/serve.rb#L108-L110
That's literally where it's crashing:
/home/simon/development/ruby/falcon_test/vendor/ruby/2.5.0/gems/falcon-0.33.11/lib/falcon/command/serve.rb:109 in `block in run'

I have a weird idea.
Is it possible you list multiple times <IP_ADDRESS> for localhost?
Can you try the following code:
Async::HTTP::Endpoint.parse("https://localhost:9292").each.map{|endpoint| endpoint.endpoint.endpoint.address}
=> [#<Addrinfo: [::1]:9292 TCP (localhost)>, #<Addrinfo: <IP_ADDRESS>:9292 TCP (localhost)>]

Yeah, it looks like it is coming up in the results multiple times:
Async::HTTP::Endpoint.parse("https://localhost:9292").each.map{|endpoint| endpoint.endpoint.endpoint.address}
=> [#<Addrinfo: [::1]:9292 TCP (localhost)>, #<Addrinfo: <IP_ADDRESS>:9292 TCP (localhost)>, #<Addrinfo: <IP_ADDRESS>:9292 TCP (localhost)>]

So looked through /etc/hosts and it looks like there were two lines for IP4 with localhost set after the .localdomain address, have removed localhost after the second entry and it has fixed the issue. So maybe something to add to the README as a potential issue, as it hasn't affected anything else until this point, so could occur on other Arch systems?

Not many systems bind to all available addresses using the localhost alias. Maybe it's why it didn't show up in other systems. But surely, it's a bug with your configuration. Maybe we can at catch Errno::EADDRINUSE and detect the situation. Feel free to submit a PR to async-io if you think it makes sense.


Clicking on page absolute coordinates

Recently one site I had automated with automate-click was updated and now there is a button you have click in time intervals. The problem is, the button has dynamic, randomly generated css class now, so there is no option to create a repeating rule for the click. So I thought, maybe it would be really cool to make the clicker able to learn and click on specific coordinates instead. The button is always on the same spot, so basically once you'd show where to click, and next time the clicker clicks on the specific spot (button) - no classes needed. Checked the FF addon catalog, didn't find anything like that (clicking on coords). That would be the real deal.

Hi @heltua,
thanks for taking the time to share your idea.
I can see how this might be useful but i think there might be issues with changing clientsite viewport/window sizes.
But i'll look into this and see if there is a way to make this a viable option.

Note:
The worst part of this ,isnt the clicking on coordinates, but changing the UI ... in a way that incorperates the input of the relative - offsets.
The simples but very ugly solution would be to just add a checkbox and 2 addional number inputs per line where the user can input a viewport-top (vh)  percentage and a viewport-left (vw) percentage.
Definetly gonna be hidden per default.
A better solution would be to toggle to switch the css selector with the coordinates ... but that requires even more UI stuff, so ... something for later maybe. (aka. read: never)

The simples but very ugly solution would be to just add a checkbox and 2 addional number inputs per line where the user can input a viewport-top (vh) percentage and a viewport-left (vw) percentage.

Basically that's what I imagined.

@heltua
Find the abolute postions of the element you want to click via
X := element.offsetLeft
Y := element.offsetTop

And place these values into the X-Click Position and Y-Click Position (hidden) columns.
Not sure how useful this is gone be, but maybe that is enough to cover your usecase.
If not, feel free to let me know about any remaining issues.
I wont promise any solutions, but i'll definitely look and think about if there is anything i can do.
Later, maybe.

@igorlogius I am very new to web development in general, could you please explain further how to get those values?
 Just dont get to used to people spoon feeding you. There is a lot of documentation on the net. I'd personally recommend https://developer.mozilla.org/en-US/docs/Web/JavaScript
There are a couple of way to get the position values. Here is one, that i think should work:

Open the Development Console (F12 or Ctrl+Shift+i )
Goto the "Console"  tab (if that is not active)
Paste onmousemove = function(e){console.log("mouse location:", e.clientX, e.clientY)}  into the prompt area and hit enter

Now when you move the mouse over the page, the Console should print the X and Y positions where the mouse is.
Hope it helps.
PS:
Positioning are window, screen size and scale depended, hope this approch works out for you.
There was a reason i did not initally include it. :-)


"$stateNotFound" debug logging trying to load Future State for the first time

I recently incorporated future states into my application, and the first time I attempt to navigate to the future state I get a log message in the console that a "$stateNotFound" event was broadcast, along with some debugging information.  However, the state transition does appear to work properly.  It is not desirable to have this information logged to the console, which gives the appearance that an error has occurred with the application.  With that said I'm not sure if there is any way to avoid this logging with better/different code, or if an enhancement to ui-router-extras may be able to take care of it.
I created a simplified Plunkr that demonstrates this -- just click on the HTML "Edit" link and observe the browser console:
http://plnkr.co/edit/KC9Hak8LOx6YmfxDTEcP?p=preview
Just for a little background, there are two main requirements that led me to the solution above:

Ability to package state definitions in a separate JavaScript file/module (the real application uses RequireJS for this.
Ability to inject runtime dependencies (services, etc) into state resolve functions.

That's simply a debug-level message.  Disable debug logging using $logProvider.debugEnabled(false);
https://docs.angularjs.org/api/ng/provider/$logProvider


Docs load really slow

Please make sure that this is a documentation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:doc_template
System information
chrome
Internet speed 120+ Mbps
TensorFlow version: all
Doc Link: all pages

It takes a really long time to load the docs. Navigating to another page can take 10+ seconds to load a new page.

@JaspervDalen It is really strange.

Do you have any Firewall? It never took more than a second or two even with low internet speeds (6 Mbps download).
Can you check with other browsers (IE, Firefox, Safari etc...)
typically how much it takes to open other websites (ex. news.google.com)

Please provide as many details as possible to find root-cause of the issue. Thanks!

@jvishnuvardhan now that I check it at home (I mostly use the tensorflow docs at work) it is a lot faster (about 3 seconds). But it is across machines (my colleagues also have the same issues) and across browsers. We haven't noticed any issues witha ny other websites. I will look tomorrow for a firewall or something.

@JaspervDalen Thanks for quick reply. Could you mention location (Europe, Asia, ...) from where you are accessing and also mention whether you notice any specific pages in TF website that are taking more time than others. thanks!

Pages with iframes can take longer to load (like the home page).
I've also noticed pages with lots of code snippets can take longer to fill in, but should start rendering quickly.
But, as Vishnuvardhan said, any specific pages and your region would be helpful it it's not an internal configuration issue.

@lamberta  It is really all pages, I can switch between all pages of the docs and all load in about 10 seconds. Today I looked at https://www.tensorflow.org/api_docs/python/tf/map_fn, https://www.tensorflow.org/api_docs/python/tf/meshgrid both really slow and https://www.tensorflow.org/api_docs/python/tf/range all took about 10 seconds. I work in Nijmegen (the netherlands), however my home is also in nijmegen (but another provider).

Ok after testing, it is mainly firefox 66.0.2 with ublock origin as add blocker (no other add-ons). In chrome it is a lot quicker (but still slow with about 2/3 seconds per loading of a page)

@JaspervDalen Last time you mentioned it took 10 seconds per page, and now it is taking 2/3 seconds per page. What did you change? Just curious. Thanks!

@jvishnuvardhan chrome is 3 second Firefox stil 10. When i tested it before with chrome it also took a long time. I think it was because i use a lot of tabs. However Firefox still has the problem

Can't reproduce but will keep an eye on site performance. Tracking an internal ticket b/133423244 for another issue.


fix possible issue where current assignees are ignored

not sure if min or max reviewers is ignored, but it's possible to "re-roll" review with someone already assigned

should be issue for https://github.com/imsky/pull-review


IMU Sensor Frequency of acquisition

Hello,
Just a simple question, I tried the Unity Scene "ImuViewSample". I would like to acquire the IMU sensor to a certain frequency (for example  60 Hz). It seems that the sensor values are updated  less quickly than this. Is it  a limitation of the sensor speed or a limitation of the C++ function HL2ResearchMode::AccelSensorLoop in  HL2UnityPlugin sln ?

Another question, how can we know the units of measurement used by the output of the IMU sensors (Magneto, Accelerometer,  Gyroscope) ? I didn't find any documentation ...

Another question, how can we know the units of measurement used by the output of the IMU sensors (Magneto, Accelerometer, Gyroscope) ? I didn't find any documentation ...

As far as i found out it is m/s rad/s and microtesla (not 100% sure about the last one)


Support path selectors

via https://github.com/fishtown-analytics/dbt/pull/2258

We might generalize this issue to: support all new selection methods + operators added in v0.17 + v0.18


Support for dotnet core?

yo aspnet nancy Test
cd Test
dotnet add package Nancy.Serialization.JsonNet

error: Package Nancy.Serialization.JsonNet 1.4.1 is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package Nancy.Serialization.JsonNet 1.4.1 supports: net40 (.NETFramework,Version=v4.0)
error: One or more packages are incompatible with .NETCoreApp,Version=v1.0.
error: Package 'Nancy.Serialization.JsonNet' is incompatible with 'all' frameworks in project '/home/chris/git/NANCY/Test/Test.csproj'.

And I'd love it to be compatible with netcoreapp1.1 as well even.  I don't know how to go about fixing this or else I'd gladly make the PR myself -- feel free to just point me in the right direction if that will get this resolved faster.

Looks like I should just move up to dotnet add package Nancy.Serialization.JsonNet --version 2.0.0-clinteastwood  however when I do that, I get a strange error on dotnet restore:
error NU1605: Detected package downgrade: Nancy from 2.0.0-clinteastwood to 2.0.0-clienteastwood. Reference the package directly from the project to select a different version.
Nancy.Serialization.JsonNet 2.0.0-clinteastwood -> Nancy (>= 2.0.0-clinteastwood)
error NU1605:  PROJECT -> Nancy (>= 2.0.0-clienteastwood)

Any idea what's up with that?
Thanks.

Any update on this issue?

I don't remember exactly what I did for this specific concern, but if it helps you at all, I do have a demo project where I use JSON in Nancy (using Newtonsoft.Json directly): https://github.com/rogusdev/docker_heroku_dotnet


docs: remove detailed description of subcommand

We don't need the long lists of usage in the README once #684 is merged.

@khaes-kth This is what I have in mind.


Allow eager instantiation of the instrumentation library

Please use the template provided below, when reporting bugs:
Expected Behavior
When mockery is used for test mocks, I don't want to see warnings about all of nyc/istanbul's modules that are required appearing prior to the first test.
Observed Behavior
Some mocking libraries hook require and by default warn when there is a required
module which has not been explicitly registered, e.g. mockery.
It is standard procedure to call mockery.enable() in a unit test prior to
requiring the code under test since obviously you want the hooks to be in place at the time that the
With lazy instantiation of the instrumenter, all of the instrumenter
library's requires generate mockery warnings at test time.
Bonus Points! Code (or Repository) that Reproduces Issue
See https://gist.github.com/evantorrie/d702ccd2e0ac2f30deb76e336128bddd

@evantorrie thanks for the thorough issue! hoping to have some time to dig into OSS this week, and will hopefully get to this. Would also happily accept a pull request.


Add a launcher

cc @cameronoelsen @sccolbert

@blink1073 Is it possible to have the launcher not in a window but in just the main area when nothing is open?

This is possible, but we'll need to expose it in Phosphide. Can we leave that as a TODO until the phosphor repo merge is done?

@cameronoelsen @sccolbert, I added #167 to track that.

LGTM


Remove boringssl dependency. Replace sha256 with absl::Hash.

Closes #1962

Codecov Report
Patch and project coverage have no change.

Comparison is base (fa07295) 92.85% compared to head (5f238a6) 92.85%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1982   +/-   ##
=======================================
  Coverage   92.85%   92.85%
=======================================
  Files         355      355
  Lines       26272    26272
=======================================
  Hits        24395    24395
  Misses       1877     1877

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

This will also fix issues found in https://github.com/chipsalliance/homebrew-verible/issues/23

Thanks, merged!


error downloading module file for private registery

http://<IP_ADDRESS>:52773/registry/packages/-/all
[{"name":"sudoku-es","repository":"","versions":["1.0.4"]},{"name":"zpipe","repository":"","versions":["1.0.8"]},{"name":"zpm-registry","repository":"","versions":["1.0.5"]}]
http://<IP_ADDRESS>:52773/registry/packages/sudoku-es
{"name":"sudoku-es","version":"1.0.4","size":3937,"hash":"7f8b78ce3bb7a1bbf6c2973d84b5ba74a495e8c4","url":"http://<IP_ADDRESS>:52773/sudoku-es/-/sudoku-es-1.0.4.tgz","versions":["1.0.4"],"versionMajor":1,"versionMinor":0,"versionPatch":4}
http://<IP_ADDRESS>:52773/sudoku-es/-/sudoku-es-1.0.4.tgz
Not Found
The requested URL /sudoku-es/-/sudoku-es-1.0.4.tgz was not found on this server.

%SYS>w $zv
IRIS for Windows (x86-64) 2020.1 (Build 215U) Mon Mar 30 2020 20:14:44 EDT
zpm: %SYS>repo -list
registry
Source:         http://<IP_ADDRESS>:52773/registry/
Enabled?        Yes
Available?      Yes
Use for Snapshots?      Yes
Use for Prereleases?    Yes
Username:       superuser
Password:
zpm: %SYS>ver
zpm 0.2.5
ZPMREGISTRY>zpm-registry 1.0.5
http://<IP_ADDRESS>:52773/registry/ - 0.0.2

I propose to fix the error by adding code to the class ZPM.Package
Method urlGet() As %String
{
SET host = ..ServerURL()
set:host'[".intersystems.com" host = host_$p(%request.Application,"/",1,*-1) //add app name #134
SET name = $ZCONVERT(..name, "O", "URL")
SET version = $ZCONVERT(..version, "O", "URL")
RETURN host _ "/" _ name _ "/-/" _ name _ "-" _ version _ ".tgz"
}

fixed


Invalid OAuth access token.

Hi Philip, i'm trying to test your framework in order to create social app authentication in my APIs.
After: curl -X POST -d "grant_type=convert_token&client_id=oQLj8tAr8QbbDw1uxC3AAmoSaIEL6IoO6ACSw9SS&client_secret=gDg2jwmQwRn33Oye8W5cB5nNRv9gCQCKN8fuIewilX6jIohQFS30F0zJclyLxOUPUnodP0jcgTxY6m1Rnjzvbek5fef3HCEMN9IJazkkM15MCkVzvwy36MBIEhvtNleT&backend=facebook&token=0647a7a063cd58d79038a57786fe4a57" http://localhost:8000/api/social/convert-token
I receive back the response:
{"error_description": "Backend responded with HTTP400: {"error":{"message":"Invalid OAuth access token.","type":"OAuthException","code":190,"fbtrace_id":"Fs2FhofZ8T4"}}.", "error": "invalid_request"}
I get client_id and client_secret from admin panel. Where do I have to get the token? I tried all alternatives: client_token from dashboard and even from https://developers.facebook.com/tools/accesstoken/
This is my facebook settings code:
# ************* Auth Social Login **************
# Facebook configuration
SOCIAL_AUTH_FACEBOOK_KEY = '926874140727083' #App ID
SOCIAL_AUTH_FACEBOOK_SECRET = '34c940f05f8d62b8f34e32f12ca4ff92' #App Secret
SOCIAL_AUTH_FACEBOOK_TOKEN = '0647a7a063cd58d79038a57786fe4a57'
# Define SOCIAL_AUTH_FACEBOOK_SCOPE to get extra permissions from facebook. Email is not sent by default, to get it, you must request the email permission:
SOCIAL_AUTH_FACEBOOK_APP_NAMESPACE = 'areaoperatoriacc'
#SOCIAL_AUTH_FACEBOOK_SCOPE = ['email']
# ************* ***************** **************

Well I got to receive:
{
"refresh_token": "9hcs4X8KOnl7M6CoSYVJjZ1RWirMA2",
"access_token": "p6XsgIuRpx45TdorRdc77PTDxQwMeA",
"expires_in": 36000,
"token_type": "Bearer",
"scope": "write read"
}
but I receive it even if I comment SOCIAL_AUTH_FACEBOOK_KEY and other parameters in settings.py!!! The only truly mandatory is SOCIAL_AUTH_FACEBOOK_SECRET
Other issue: Have I to use App Token or User Token shown in https://developers.facebook.com/tools/accesstoken/ page???
It works only with User Token!

You don't need SOCIAL_AUTH_FACEBOOK_TOKEN in your settings.py.
You are using a wrong token, that is why it isn't working. The token you are looking for is way longer and given by facebook when you authenticate on a client app (for example an iOS app or Js web app).
You need to use the access token shown in User Token on the https://developers.facebook.com/tools/accesstoken page.
Do you have any more questions ?

Yes, probably because of my bad understanding of english language, sorry:

Definitively, WHERE do I find
SOCIAL_AUTH_FACEBOOK_KEY = 'xxxxxxxx' #App ID
SOCIAL_AUTH_FACEBOOK_SECRET = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' #App Secret
values to populate settings.py variables?

When and where I have to use 'Application' Model values in the registration/login process? When Facebook Dashboard values?

Thank you so much!

You find these values on your facebook application dashboard. Normally, the app ID is shown and you have to enter a passwird to display the app secret.
You need to create an application model as shown in the readme and use its values when authenticating. Basically, you have to copy client_id and client secret. The token is given by facebook and you can get the test token shown under User Token on the https://developers.facebook.com/tools/accesstoken page.

Only a last question:
I'm writing an application BE, to let Android, WP and iOS devices register, login and so on through Facebook, Twitter and Google accounts.
Does your package manage these features automatically?
Where can I find examples?
Thanks again!

It does not handle all of this automatically but it helps a lot.
Everything is in the readme. You should be able to setup everything based solely on the readme.
There is the facebook example on the readme. I don't have examples of clients you'll have to do the research yourself but I believe it is fairly simple.


[Ticket][PMT] [Testing] Polish GeorgiaCORE

Non-Profit Description
GeorgiaCORE is an application where users can create customized chat-bot interfaces by building out trees showing possible paths of navigation in the chatbot.
Ticket Description
This ticket is slightly different from past tickets. Effectively, we want you as the developer to perform what is typically called QA Testing in the real world (short for quality-assurance testing). In two weeks, we will be meeting with the product team for GeorgiaCORE to fully handoff the product, and we would like you as the developer to try your best to break the app in anyway possible and either document or fix it (more preferred).
This means, we want you to play around with everything from the user access page, to the actual chat navigator, to the tree system and intentionally try to do malicious actions (in the scope of a user). To document issues you can't resolve, you can write comments below.
Acceptance Criteria

[ ] Break the app and document the breaks or fix it

Other Notes

Please look at the GeorgiaCORE Setup section in the ReadMe to understand how to work on the project, we also have it linked in the Slack channel
Ask Manu or Jay for help on anything here if needed! (We both worked on this project last semester!)

Supporting text inputted into tree under "questions" category not displayed in actual chat navigator.
When locked, user is still able to add new nodes by clicking on the dot and dragging it to the blank canvas.
Adding a question with length > 65 cuts it off in the chat navigator and user cannot see entire question that was inputted into the tree system. Likely because in the Tree System there is no max on how many characters can be inputted into the question field (see image)
Adding a long answer choice after inserting a question node leads to entire answer choice text not being displayed in chat navigator (see image).

Adding an icon for one of the answer choices for a question and setting other question's icon to no icon leads to both answer choices having no icon in the chat navigator preview.
Deleting a parent node of a leaf does not delete its children
Last Saved: User needs to go back to home page and re-open the tree system in order for last saved time/date to update, it doesn't update automatically if user edits the tree system page and then clicks "save"
Adding a Text node and entering a long question in the first field leads to text title overlapping with other elements on the Chat Navigator(see image):

URL Field in Link Node not required -> leaving it empty does not enable user to click on anything in chat navigator

Fixed issues at pull request #85


Enhance search by filtering by URL, hostname and dns resolve server

⚠⚠⚠ Since we do not accept all types of pull requests and do not want to waste your time. Please be sure that you have read pull request rules:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md#can-i-create-a-pull-request-for-uptime-kuma
Tick the checkbox if you understand [x]:

[x] I have read and understand the pull request rules.

Description
Fixes #5422 . This PR is pretty simple (just three lines enhancing existing filters), but makes so much sens to me..
Note: Even if this is not supposed to occurs, the code handle the case url would be null/undefined.
Type of change

New feature (non-breaking change which adds functionality)

Checklist

[x] My code follows the style guidelines of this project
[x] I ran ESLint and other linters for modified files
[x] I have performed a self-review of my own code and tested it
[ ] I have commented my code, particularly in hard-to-understand areas (including JSDoc for methods)
[x] My changes generates no new warnings
[ ] My code needed automated testing. I have added them (this is optional task)

Screenshots

Hi @CommanderStorm! After some tests, it appears regex is more efficient when you have >= 20 monitors. Before, lower-case is better. But because UK is designed to handle a lot of monitors, regex has been used instead. Thanks for the suggestion!!

I was thinking more along the lines of something like this:
// filter by search text using regex
// matches monitor name, tag name, or tag value
let searchTextMatch = true;
if (this.searchText !== "") {
    try {
        const regex = new RegExp(this.searchText, "i"); // "i" for case-insensitive matching
        searchTextMatch =
            regex.test(monitor.name) ||
            (monitor.url && regex.test(monitor.url)) ||
            (monitor.hostname && regex.test(monitor.hostname)) ||
            (monitor.dns_resolve_server && regex.test(monitor.dns_resolve_server)) ||
            monitor.tags.some(tag => regex.test(tag.name) || (tag.value && regex.test(tag.value)));
    } catch (e) {
        console.error("Invalid regex pattern:", e);
        searchTextMatch = false;
    }
}

Relevant indeed, I forgot to take into account the null/undefined values. Will do the changes, thanks @homelab-alpha

Or better: suggest the change via the review feature so you can appear in the contributors.

@Ionys320 , You came up with the fix, we can share the credit. 😊

So, the input is now regex friendly, that's fun. I wonder if it can broke something, I tried do search a fiew fields and it went fine.
The only issue I could fine is about . : when you search for goo.gle, it will find google.fr, even it it's not the request (logic because . is considered as any character). I wonder if it's an issue, because if so, we'll need to rollback to the lower-case method. @CommanderStorm?

@Ionys320, Can you show it in a screencast what you mean? Because what you're saying, I can't recreate.

I'd consider reflexes working somewhat like a feature.
Alternatively, we could just replace special characters away by escaping them.
I am fine with both

Approved then!

Okay so: the searchbar is now escaping regex expression (https://github.com/louislam/uptime-kuma/issues/5466#issuecomment-2558782685).
This PR creates the issue #5466. This issue occures because the PR extends the used fiels, but the PR is not creating the issue neither. But maybe fix #5466 before merging the PR would be a good move.


Delay call to strlen in filename_filter

The call to strlen in ignore.c:344 could be delayed until the while loop on line 395 where the code path should have become "cooler".  I have not benchmarked it though.

Nice catch, though I'm almost certain that the compiler is smart enough to move the strlen() down.


Cross origin request failing for Leap library

Hello -> On attempting to load the snapmotion leap extension,  I get an error in the console.
Although this LOOKS like an error in the crossorigin.me project, I am submitting the issue here, in case it's snapmotion-specific.
snap.html#open:https://raw.githubusercontent.com/technoboy10/snapmotion/master/leap.xml:1 XMLHttpRequest cannot load http://crossorigin.me/http://js.leapmotion.com/leap-0.6.4.min.js. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://snap.berkeley.edu' is therefore not allowed access.

Update:  if i add a link to a local copy of the leap library to the snap.html in a local version of snap (and get rid of the http: call in the initializer block, the rest of the blocks run fine.  This suggests that the ONLY problem is the cross origin issue.


option to build libtcod statically?

It's currently a bit of a pain to run examples and applications using the library at the moment due to the libtcod.so dependency at run time.
I think statically linking the internal libtcod (e.g. as a .a) could avoid this and make the resulting executables self-contained, of course apart from system libraries such as SDL.

I agree, this would be a great usability win.
Unfortunately, I don't really have the time and knowledge to investigate this at the moment. Do you? I'd be happy to help in any way I can and I would love to accept a pull request that implements it.

Sure!
I can't promise how soon, but I'm willing to look at this.

I looked into it, it's easy enough to make it link to an .a instead of an .so, as well as to add println!("cargo:rustc-link-lib=dylib=SDL2"); to specify SDL2 as a dependency for linking libraries.
However, it looks like rust insists on passing the order of the linker arguments as "-l" "SDL2" "-l" "tcod" which is the wrong way around, so I get screens full of symbol errors. Nothing that I tried could make it work.

The PR above is merged, I'm closing this issue. Thanks so much for doing this @laanwj!


Regex shift

This P.R. mainly refers to:
Shifting of regex module from Javascript to Haskell part
Addition of a function to save raw error message generated by ghcjs compiler
I have used subRegex function from Text.Regex library in order to replace regex patterns in Haskell. Also, the error message is stored in two files, one(hash.err.txt) here filtered message is saved and the other one(hash.rawerr.txt) the error message generated by ghcjs.

Done the changes suggested. I tried to fix those commits with unknown-email-address but I am unable to do so. Tried to work on that with the help of git rebase but, it is not listing those commits in the list of rebase(might be because they are the merge branch commits).
Is there any other way to approach them?

I haven't tried it myself, but the accepted answer at https://stackoverflow.com/questions/9930637/edit-a-merge-commit-with-git-rebase has steps to amend a merge commit.

After rebasing the commits it updates all the commits till head, and I pull the master so that's why it updated your changes too(which have been already added in the master) simultaneously. It has became much clumsy so, it is a better idea to put a new Clean P.R., I have already shifted the changes to a new branch with proper author and everything. Once this P.R. is closed I'll put the updated one.
I think this is a better idea to go.

Okay, a new pull request sounds fine.  I'm happy with this change as-is.


Added Procedure to Split Dataframe using Groupby

I've added a procedure for Splitting a Dataframe using the Groupby procedure in Pandas. Having talked to a few other CPA's who were doing this type of process manually, I figured I'd put together a short tutorial.

Thanks for the contribution! Please add more if you are up for it!

Thanks - I intend to. I have a bunch of random python stuff, that I working to organize and make usable by a larger group of people.


HDDS-8931. Allow EC PipelineChoosingPolicy to be defined separately from Ratis

What changes were proposed in this pull request?
A cluster may have set the PipelineChoosingPolicy to the HealthyPipelineChoosePolicy for Ratis, but it adds overhead and is not necessary for EC, as the EC pipeline is always healthy:
  public boolean isHealthy() {
    // EC pipelines are not reported by the DN and do not have a leader. If a
    // node goes stale or dead, EC pipelines will by closed like RATIS pipelines
    // but at the current time there are not other health metrics for EC.
    if (replicationConfig.getReplicationType() == ReplicationType.EC) {
      return true;
    }
    ...

To allow for flexibility, add a new config for the ECPipelineChoosingPolicy so it can be different from the Ratis policy.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8931
How was this patch tested?
Modified and added to the tests in the PipelinePolicyFactory.

Thanks @sodonnel for the patch.


Update autoprefixer

hello, can you bump the autoprefixer version to the latest 7.1.4?
Thanks
https://github.com/sindresorhus/broccoli-autoprefixer/blob/master/package.json#L33

^7.0.0 will install 7.1.4. You can read more about it here https://nodesource.com/blog/semver-tilde-and-caret/. Unless there's a critical bugfix I don't see the point in bumping it.


Bed filter update

Some updates to the bedFilter cmd. Cmd has been modernized to use chans for reading and writing (previously held output entries in memory). Options have been organized into a settings struct. Two new options: minNameFloat and maxNameFloat, for filtering entries by value when using the name field to store floats.
Also took this opportunity to write tests for this function.

This seems to have a lot of the acceleration code. Since that PR is marked as 'do not merge' should this PR be reviewed yet?

I am seconding Dan's comments about the acceleration code.  Should we review the acceleration code, or should this also be "do not merge" for right now?


TSM_0064 failed

With latest changes in JsonPreprocessor the self test case TSM_0064 fails.
TSM_0064 is a BADCASE test of variant handling: A local config file is passed to command line parameter config_file.
Because of a local config file is not a full configuration file, this error occurred previously:
UNKNOWN - Could not set variable 'params['global']['teststring_bench']' with value 'I am the 'bench1' configuration of tsm-test'! Reason: name 'params' is not defined
This error is changed now to:
UNKNOWN - Exception occurred.
I would prefer to have the previous error message back.

Hello Holger,
The error message from JsonPreprocessor (JPP) is changed in this case because our new requirement:
•	In JPP ticket 182:

We decided to set root key in case it’s not available

Hi Thomas, hi Son,

while updating the documentation I continued with playing around with possibilities. Outcome: Several issues need to be fixed:

(1)
${testdict.subKey.subKey.paramA} : "DEF"
returns: Could not set variable ... Reason: name 'testdict' is not defined'

NOT OK

It means now if JPP loads a jsonp file like the file "config_file: ./localconfig/tsm-test_localconfig_bench1.jsonp" in TSM_0064 which contains:

{
   ${params}['global']['teststring_bench'] : "I am the 'bench1' configuration of tsm-test"
}

Now, JPP does not raise error message “Could not set variable ... Reason: name 'params' is not defined'” any more, instead the TestsuitesManagement package will raise the error message that the required property is not in configuration file:

[ ERROR ] The parameter 'Project' is a required property in configuration file C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles\localconfig\tsm-test_localconfig_bench1.jsonp.

In this case I think we should update the verified log message in selftest.
Thank you,
Son

Hello Holger,
I just run and get error message at the top of test suite like below:
C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles>%RobotPythonPath%\python -m robot.run --variable config_file:"C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles\localconfig\tsm-test_localconfig_bench1.jsonp" tsm-testfile-01.robot
==============================================================================
Tsm-Testfile-01
==============================================================================
[ ERROR ] The parameter 'Project' is a required property in configuration file C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles\localconfig\tsm-test_localconfig_bench1.jsonp.

Test Case tsm-testfile-01 :: tsm-testfile-01                          | UNKNOWN |
Parent suite setup unknown:
Exception occurred.
------------------------------------------------------------------------------
Tsm-Testfile-01                                                       | UNKNOWN |
Suite setup unknown:
Exception occurred.

1 test, 0 passed, 0 failed, 1 unknown
==============================================================================
Output:  C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles\output.xml
Log:     C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles\log.html
Report:  C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles\report.html

C:\Users\mas2hc\Desktop\data_storage\workspace\ROBFWAIO-OSS\robotframework-testsuitesmanagement\test\testfiles>

Thank you,
Son

Hi Son,
what you list above, is the console output. But the self test is based on a comparison of the log files. And in the log file the error message is not seen.
What have you done to reproduce the error I described? In your console listing I miss the debug output file. Therefore this cannot be a TSM self test result. Have you executed TSM_0064? Or have you executed the affected robot file manually? This would make no sense! You have to execute TSM_0064. Then you would have a debug log file. And this file only contains
UNKNOWN - Exception occurred.
and not your detailed error message.
For me the question is: Why is this message not logged?

OK, this is the error message:
[ ERROR ] The parameter 'Project' is a required property in configuration file C:\...\tsm-test_localconfig_bench1.jsonp.
And is this the code, that causes this error message?
logger.error(f"The parameter {error.message} in configuration file {self.sTestCfgFile}.\n")
Hmm, this way of combining strings with parameters to generate an error message, is really interesting ;-)
Please add single quotes around {self.sTestCfgFile}.

Hello Holger,
I updated the error message log as your comment, and added into the pull-request from stabi branch to develop branch.
Thank you,
Son

Hello Holger,
logger.error is error log mechanism of Robot Framework.
Previously, we can observe error message in the text log file because the error message came form raise Exception(<error message>) in JsonPreprocessor package.
Now, due to the requirement changed, the jsonp file in the TSM_0064's test scenario do not cause any Exception. Then instead of exception from JPP, the error massage is raised by logger.error from Robot test suite.
Thank you,
Son

Hi Son,
OK, clear to me now. But the problem still is:
Instead of a speaking error message we have an "UNKNOWN - Exception occurred." in the log file. And this does not help anyone.
We need a speaking error message for at least two reasons:

Users need to be informed
An automated self test needs to be supported

All like before!
What do you plan now to solve the problem?

Hello Holger,
For this, we have to modify Robot Framework core to to update error message to UNKNOWN state instead of UNKNOWN - Exception occurred..
Cuong will handle this implementation :)
Thank you,
Son

@milanac030988 : Can you please look into this issue?
@HolQue : in this case we need to adapt as intermediate solution the testcase to the new error-message.

Hi @milanac030988
before you start please take a look at my proposal at the end of
https://github.com/test-fullautomation/robotframework-testsuitesmanagement/issues/83#issue-1532185345

I already adapted the error message to the new official one. But intermediately I will change this to
Exception occurred.
Update of pull request soon.

Hi Son,
I still try to understand, but I fail. We can discuss directly at Monday.
For now only some observations:
CConfig.py
nearby
if bJsonSchema:
I see line
BuiltIn().unknown('JSON schema validation failed!')
But I can see this message neither in console nor in debug log file. Why this? Or asked other way round: What do I have to do to see this message anywhere in log or console?
I added an own message
if bJsonSchema:
    BuiltIn().log(f">>>>>>>>>>>> test output:")

Same result: not logged.
I tried:
if bJsonSchema:
    BuiltIn().log(f">>>>>>>>>>>> test output:", "ERROR")

Now the message is visible in console (because of trace level "ERROR"), but still not in debug log file.
I exchanged
logger.error(f"The parameter {error.message} in configuration file '{self.sTestCfgFile}'.\n")
by
BuiltIn().log(f"(--2--) The parameter {error.message} in configuration file '{self.sTestCfgFile}'.\n", "ERROR")
Same outcome: visible in console, but not in debug log file.
You use logger.info() at some positions. I also added an own logger.info
logger.info(f">>>>>>>>>>>> logger.info test output")
at the beginning of loadCfg().
Same result: Not seen anywhere.
I am really confused now, because this is in opposite to all what I experienced while I reworked the logging some weeks ago. Every BuiltIn().log(..) has been logged to debug log file. Why not here?
I switched to CSetup.py where loadCfg(...) is executed, and added
BuiltIn().log(f">>>>>>>>>>>> test message")
at the beginning of testsuite_setup()
Outcome: Test message not seen in console (OK, because trace level is default "INFO", and this level is not printed to console), but is written now to debug log file!
After this I added
logger.info(f">>>>>>>>>>>> logger.info test output 2")
Also this info is written to debug log file.
This is really interesting. Why do BuiltIn().log() and logger.info() work like expected at this position, but not inside loadCfg(...)? What is the difference?
In CSetup.py I also see this:
if not RobotFramework_TestsuitesManagement.CTestsuitesCfg.oConfig.bLoadedCfg:
    BuiltIn().unknown(CConfig.sLoadedCfgError)
    return

I extended to
if not RobotFramework_TestsuitesManagement.CTestsuitesCfg.oConfig.bLoadedCfg:
    BuiltIn().log(f">>>>>>>>>>>> bLoadedCfg: {RobotFramework_TestsuitesManagement.CTestsuitesCfg.oConfig.bLoadedCfg}")
    BuiltIn().unknown(CConfig.sLoadedCfgError)
    BuiltIn().log(f">>>>>>>>>>>> test message after unknown")
    return

And the debug log file tells me:
INFO - >>>>>>>>>>>> bLoadedCfg: False
UNKNOWN - Exception occurred.

Very good: BuiltIn().log() works; the bLoadedCfg INFO is logged.
Now I want to see the error message and tried:
if not RobotFramework_TestsuitesManagement.CTestsuitesCfg.oConfig.bLoadedCfg:
    BuiltIn().log(f">>>>>>>>>>>> bLoadedCfg: {RobotFramework_TestsuitesManagement.CTestsuitesCfg.oConfig.bLoadedCfg}")
    BuiltIn().log(f">>>>>>>>>>>> sLoadedCfgError: '{CConfig.sLoadedCfgError}'")
    BuiltIn().unknown(CConfig.sLoadedCfgError)
    BuiltIn().log(f">>>>>>>>>>>> test message after unknown")
    return

But unfortunately the debug log file tells me:
INFO - >>>>>>>>>>>> sLoadedCfgError: ''
Hmm, nothing inside CConfig.sLoadedCfgError? Why not? Where is the error message gone?
I tried with an own error message, to verify what BuiltIn().unknown() is able to do at this position:
if not RobotFramework_TestsuitesManagement.CTestsuitesCfg.oConfig.bLoadedCfg:
    BuiltIn().log(f">>>>>>>>>>>> bLoadedCfg: {RobotFramework_TestsuitesManagement.CTestsuitesCfg.oConfig.bLoadedCfg}")
    BuiltIn().log(f">>>>>>>>>>>> sLoadedCfgError: '{CConfig.sLoadedCfgError}'")
    # BuiltIn().unknown(CConfig.sLoadedCfgError)
    BuiltIn().unknown("This is my own error message")
    BuiltIn().log(f">>>>>>>>>>>> test message after unknown")
    return

Outcome: A BIG HOORAYY!!
My own error message is seen in console and in debug log file:
UNKNOWN - This is my own error message
Conclusions:

I admit, I do not really have much knowledge of the code. But I am very worried about my observations.
Intuitively I would expect that is should be possible to log a message and then react on what happened (e.g. set the rest result to UNKNOWN). I cannot beliefe that it would be required to change the code of the RobotFramework core to enable this.
I would assume now, that it is possible to solve the log problem easily e.g. by returning a non empty sLoadedCfgError inside testsuite_setup and log this with
BuiltIn().unknown(CConfig.sLoadedCfgError)
But still the question is why it is not possible to do this inside loadCfg directly.

Hello Holger,
I pushed new commit c137516797 to the stabi branch to fix the issue based on test case TSM_0064.
Could you please help me update log template again for TSM_0064?
Thank you,
Son

Hi Son,
TSM_0064 works now!
But your code change has an impact on further self test cases, e.g. TSM_0050.
Previous error version:
ERROR - The parameter 'WelcomeString' is a required property in configuration file ...
UNKNOWN - JSON schema validation failed!

Actual error version:
ERROR - The parameter 'WelcomeString' is a required property in configuration file ...
FAIL - KeyError: 'WelcomeString'

In my opinion the first error version is the better one. Because the first line tells about what exactly went wrong. And the second line tells about the impact (the failed validation). To know about the impact is helpful for users.
Whereas in second error version the second line "FAIL - KeyError: 'WelcomeString'" is not really helpful for users. It's also mainly redundant to the first line.
I would prefer to have the first version back. Maybe in one line only.
Next aspect:
The fact that a parameter is required, is not the error. The error is that a required parameter is missing. The first line in error message is a bit incomplete.
Summarized, my proposal is:
Error: Required parameter '...' is missing in configuration file '...'. JSON schema validation failed!
Finally:
The test result is FAIL now, but expected is: UNKNOWN (like before).

Hello Holger,
Thank you for your quick feedback!
By mistake, I deleted the line of code that raise Unknown state, I added it again and updated the error message log as your suggestion in latest commit on stabi branch.
Thank you,
Son

Hi  @namsonx ,
what is the status of this issue?
Thank you,
Thomas

Hello Thomas,
This was fixed, and run success on Gitlab.
Thank you,
Son

Retest successful. Issue can be closed.


Refactored to use Menu.ROOT_MENU_ID (as in MenuList)

@forest-tong :+1: Congrats on your first Commit!


Touch controller support.

Greetings,
I was wondering about touch controller support, specifically for the XPT2046.
I see that there are some touch functions outlined in the bb_spi_lcd.h file, but they have no implementations at the moment.
I know that i could use a third party library for just the touch handling, but I am very tight on free pins, so I was hoping to have the touch controller share the same pins as the lcd (aside from the CS pin of course).
I have attempted to do something similar to how TFT_eSPI implements this, but so far I havent had any success.  Could you guide me a bit on this issue?  How would one go about implementing device switching and clock speed changing while using the DMA.  Btw, I am using an esp32-cam.
Many thanks for this library and all your work with it and JPEGDEC.

I removed the touch support a while ago because I was trying to shrink the code to still fit on the ATmega328. I didn't add it back because it seemed like an unnecessary part of this library. I just went looking for it and couldn't find it. If you can do a PR to add it back, I'll accept it.


mongodb uri : {mongoUri} sems to be an issue with bold display

It seems that the Mongo uri should be displayed in bold on the modal launched after the mongo service is launched.
Here is the screen of what I get today :

corrected by InseeFrLab/helm-charts-datascience#58


What are long_run and short_run trend indicators?

Hi,
What are long_run and short_run trend indicators?
I use it. And print results gor long_run and short_run as DataFrame. But i did not difference between results. I did not understand what is benefits those indicators?
long_run results Bitcoin 5 minute data
close
0   37056
1   37056
2   37075
3   36883
4   37144
..    ...
75  35389
76  35097
77  35173
78  35340
79  35458
[80 rows x 1 columns]
long_run results Bitcoin 5 minute data
close
0   37056
1   37056
2   37075
3   36883
4   37144
..    ...
75  35389
76  35097
77  35173
78  35340
79  35458
[80 rows x 1 columns]

Hello @coinrazzi,
They are part of my custom indicators along with amat and aobv. In fact long_run and short_run are required for amat and abov. I ported Archer Moving Average Trend (amat) from my Trading View Pine Script Trade Archer - Moving Averages - v1.4F).
Regarding long_run (and conversely short_run), their fast and slow parameters work best with smoothed series. It makes it easier to tell if they are increasing and decreasing. Long Run then checks if fast and slow are potentially crossing or both increasing (think parallel):
def long_run(fast, slow, length=None, offset=None, **kwargs):
    """Indicator: Long Run"""
    # ...

    # Calculate Result
    pb = increasing(fast, length) & decreasing(slow, length)  # potential bottom or bottom
    bi = increasing(fast, length) & increasing(slow, length)  # fast and slow are increasing
    long_run = pb | bi  # more signals with or, less with and (&)

    return long_run # Returns binary series 1 if long trend, 0 otherwise

Archer Moving Average Trend (amat) just uses Moving Averages for their fast and slow parameters. But you are not limited to just Moving Averages as fast and slow parameters of course, preferably smoothed series. For instance, you can pass in two different Moving Averages like HMA_10 and SMA_50 for fast and slow or even macd and macd signal for fast and slow. Experiment.
Long Run and Short Run return binary series<PHONE_NUMBER>111...), 1 or a trend and 0 not a trend.
Example:
import pandas_ta as ta

df = # ohlcv
closedf = df["close"]
# This is literally **amat**
lrun = df.ta.long_run(ta.ema(closedf, 8), ta.ema(closedf, 21), append=False)
srun = df.ta.short_run(ta.ema(closedf, 8), ta.ema(closedf, 21), append=False)
srun.tail(50).plot(kind='bar', figsize=(16,0.5), color=['red'], linewidth=1, alpha=0.45)
lrun.tail(50).plot(kind='bar', figsize=(16,0.5), color=['green'], linewidth=1, alpha=0.45, title=f"{lrun.name}(green) & {srun.name}(red)")

Hope this makes more sense.
Kind Regards,
KJ

Great information. thank you :)


feat: a simple nextjs project added, uses api and dynamic routing

A simple, original Next.js project added which shows how to consume an API and implement dynamic routing.

@akash0708 please remove package-lock.json file from the commit history and add it into .gitignore.


Fix iOS 11 WebView extra white space on top due to contentInsetAdjustmentBehavior

Same IOS 11 issue as https://github.com/facebook/react-native/pull/15023
Fixes the annoying 20px content insert on top.

Motivation
Fix the behaviour of WebView when building the iOS project with Xcode 9 due to the contentInsetAdjustmentBehavior.
Test Plan
Expected Results:

Screenshot of the issue (Building with Xcode 9)

Release Notes

[IOS] [BUGFIX] [WebView] - Fix extra white space added to webView due to iOS 11 contentInsetAdjustmentBehavior

@shergin screenshots and #if gating updated.


[Bug] system-lib-prefix would be cleared if device is not strictly android while mlc_llm compile

🐛 Bug
Hi there, thank you so much for your great work. I am new here, sorry if this is silly.
I have met this problem while converting a LLM for a GPU enabled Android device via Python APIs.
While in mlc_llm compile stage, if I set device to android:adreno (I assume it won't invoke Adreno GPU if I simply set it to android, right?), it will trigger this code and clear the system-lib-prefix.
For my own case, I would just comment them out. But in my opinion, this logic does not make sense, and that's why I am submitting this report.

passing in android would be fine, and it will invoke adreno GPUs.

After reading through the code, i think @BitCircuit you are right and we can safely remove this line, do you mind send a PR?


Set Solr timezone to UTC when converting local date/time to the stored value that assumes Zulu (Z)

References

Fixes #8558

Description
When a date is formatted for storage in Solr, the assumption of UTC in SOLR_DATE_FORMAT requires SOLR_TIME_ZONE to also be set so the input date is correctly converted from the default local timezone.
Instructions for Reviewers
Set SOLR_TIME_ZONE for SimpleDateFormat object returned from SolrUtils.getDateFormatter(). This method is used to format data/time for:

oai item.lastModified
search lastModified, lastIndexed, _dt date fields
authority creation_date, last_modified_date

Tests:

Update an item so that its last modified time is current, then refresh the item's oai (oai import) or search (index-discovery) entry. The "last modified" date/time values should have been converted from the local timezone to UTC
Create a new authority record (e.g., input new name in submission form with ORCID authority enabled), then check the creation_date has been converted to UTC

Checklist
This checklist provides a reminder of what we are going to look for when reviewing your PR. You need not complete this checklist prior to creating your PR (draft PRs are always welcome). If you are unsure about an item in the checklist, don't hesitate to ask. We're here to help!

[X] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & integration tests). Exceptions may be made if previously agreed upon.
[X] My PR passes Checkstyle validation based on the Code Style Guide.
[X] My PR includes Javadoc for all new (or modified) public methods and classes. It also includes Javadoc for large or complex private methods.
[ ] My PR passes all tests and includes new/updated Unit or Integration Tests based on the Code Testing Guide.
[ ] If my PR includes new libraries/dependencies (in any pom.xml), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
[ ] If my PR modifies REST API endpoints, I've opened a separate REST Contract PR related to this change.
[ ] If my PR includes new configurations, I've provided basic technical documentation in the PR itself.
[ ] If my PR fixes an issue ticket, I've linked them together.

Added this change to the DSpace 7.5 instance I am running, and this has fixed the OAI-Index issue for us (item lastmodified was ending up at NZT +12 which meant it was excluded from any OAI requests for 12 hours).


Make it async

@oleynikandrey hello! Looks like this is not backwards compatible.
I suppose we can drop Python 2.6 support, but I'd like to keep compatibility with 2.7.
And what if someone uses cent in non-async environment like Django?

Thanks for the tip about wrong key used history method!

@oleynikandrey what do you think? Maybe this should me another project? I can add a link to your Github account and async client into docs

@FZambia, sorry I've opened this pull request accidentally)
This definitely should be another project. But It won't be compatible with python 2.7, because of aiohttp  require Python 3.5.3+.
I'll create separate project and send link.


Support trailing comma in object literal

This PR add support for trailing comma in object literal.
 $ jq '{foo,bar,}' <<< '{"foo": 1, "bar": 2, "baz": 3}'
{
  "foo": 1,
  "bar": 2
}
 $ jq -n 'module { x: 1, }; .'
null

Thank you for the contribution!
Hmm, it's interesting that jq doesn't support the trailing comma in bind patterns while supporting it on other places :thinking:


ENH: Update submodules to the latest master branches

The submodules were out of date and are causing problems wiht reporting
dirty modified content.  Fixed by reseting the the master branch of each
repository submodule.

FYI:  I don't understand how the submodules work in this example.  They always seem to cause
On branch fix-sphinxexamples-build-20190314
Your branch and 'origin/fix-sphinxexamples-build-20190314' have diverged,
and have 2 and 3 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   breathe (new commits)
	modified:   gitstats (new commits)
	modified:   sphinx-bootstrap-theme (new commits)

When not syncronized with the master branch.

If I recall correctly, there is an incompatibilty with breathe master and ITK Doxygen XML.

How do I avoid the git messages that constantly state that my repository is out of date due to the submodules?

git submodule update --init --recursive ?

I tried that it did not seem to work.
Sent from my iPhone

On Mar 15, 2019, at 12:04 PM, Matt McCormick<EMAIL_ADDRESS>wrote:
git submodule update --init --recursive ?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

Fixed in a different commit.


[ut] unit test

Codecov Report

Merging #73 into develop will increase coverage by 0.25%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           develop    #73      +/-   ##
=========================================
+ Coverage    99.74%   100%   +0.25%
=========================================
  Files           20     20
  Lines          389    389
  Branches        33     33
=========================================
+ Hits           388    389       +1
+ Misses           1      0       -1

Impacted Files
Coverage Δ

src/state-machine/state-machine.ts
100% <0%> (+0.65%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cfef59f...a824aaf. Read the comment docs.


How to import external css font-face in vite lit element

Which package(s) are affected?
Lit Core (lit / lit-html / lit-element / reactive-element)
Description
How to import external css font-face in vite lit element.
I'm trying to put font family helvetica-neue-bold.woff and woff2 files in locally and import them as
export default css  @font-face {     font-family: "helvetica-neue-bold."; /*a name to be used later*/     src: url("./fonts/helvetica-neue-bold..woff"); /*URL to font*/   }
and trying to use
.eyebrow-2 {
font-family: "Helvetica Neue", "-apple-system", "system-ui",
"BlinkMacSystemFont", serif;
font-size: 10px;
font-weight: 200;
line-height: 13px;
letter-spacing: 2px;
}
but it is not working. Could you please help me with my issue?
files
Reproduction
How to import external css font-face in vite lit element.
I'm trying to put font family helvetica-neue-bold.woff and woff2 file in locally and import as
export default css  @font-face {     font-family: "helvetica-neue-bold."; /*a name to be used later*/     src: url("./fonts/helvetica-neue-bold..woff"); /*URL to font*/   }
and trying to use
.eyebrow-2 {
font-family: "Helvetica Neue", "-apple-system", "system-ui",
"BlinkMacSystemFont", serif;
font-size: 10px;
font-weight: 200;
line-height: 13px;
letter-spacing: 2px;
}
but it is not working. Could you please help me with my issue.
files
Workaround
export default css  @font-face {     font-family: "helvetica-neue-bold."; /*a name to be used later*/     src: url("./fonts/helvetica-neue-bold..woff"); /*URL to font*/   }
and trying to use
.eyebrow-2 {
font-family: "Helvetica Neue", "-apple-system", "system-ui",
"BlinkMacSystemFont", serif;
font-size: 10px;
font-weight: 200;
line-height: 13px;
letter-spacing: 2px;
}
but it is not working. Could you please help me with my issue.
files
Is this a regression?
No or unsure. This never worked, or I haven't tried before.
Affected versions
"lit": "^2.7.5"
Browser/OS/Node environment
MacOS

Hello. Im having the same issue. I can only import from outside the element, but not from inside. how solve that?


[paperless] chore: switch to paperless-ngx project

Description of the change
This switches to the community fork of paperless-ng. The tracking ticket in paperless-ngx is https://github.com/paperless-ngx/paperless-ngx/issues/68.
Please see the following references for the community discussion of the decision to fork:

https://github.com/jonaswinkler/paperless-ng/issues/1599
https://github.com/jonaswinkler/paperless-ng/issues/1632.

Benefits
Switch to the community maintained paperless-ngx fork.
Possible drawbacks
None that I am aware of.
Additional information
This is a non-breaking change for the paperless application itself, but as it changes the upstream, I've opted to bump the major version.

Checklist

[x] Title of the PR starts with chart name (e.g. [home-assistant])
[x] Chart version bumped in Chart.yaml according to semver.
[x] Chart artifacthub.io/changes changelog annotation has been updated in Chart.yaml. See Artifact Hub documentation for more info.
[x] Variables have been documented in the values.yaml file.

Can someone please advise me how to correctly update the README? I can't find documentation for it, although it suspiciously looks like helm-docs.

Can someone please advise me how to correctly update the README? I can't find documentation for it, although it suspiciously looks like helm-docs.

No need to manually update the README, we auto-generate that upon merging the PR. Any manual modifications will be lost then.
I will take a look at the PR later tonight 😃

@bjw-s that's good to know, thanks! How can I take care of updating the Source Code section then? I didn't find it anywhere else and updated it directly in the README.md for that reason (https://github.com/k8s-at-home/charts/pull/1436/files#diff-2e548bd16ddaf26ec88fbd01ce605997e61c43301e73c5a723aaf90a21908156R11)

@morremeyer - that is in at https://github.com/k8s-at-home/charts/blob/ab59fc8c6e58d0146c052086a07438dad5fb6389/charts/stable/paperless/Chart.yaml#L12
As user of paperless and contributor to the k8s-at-home I am really glad to see the project moves on. So no reason for not switching the chart to the new community fork.

@bjw-s Thanks for the review. I overlooked that you added it as a suggestion, so I updated it locally.