Bridge to Stellar not working

Yesterday I initiated three separate transactions to bridge funds from tfchain to Stellar. None have arrived (nor have the funds been refunded).
Initiating account on tfchain: 5HW86mnxp4JBpQrASKbHcJPf2Mxmma8CuiB5azZZJq32a38c
Destination account on Stellar: GAGUIASHDZZC6LOPKZFCEMU37YSVYV2EAS5TLSE2VGCNOMHIEW2BK6O5
Amounts are: 101, 101, 2
First two were initiated through the Dashboard and the third through Polkadot UI. The amounts were deducted from tfchain balance in each case.

bridge is stuck indeed, might be related to the tfchain runtime upgrade.
It's in a crash loop, it finds the latest tfchain block and crashes here

Thank you @scottyeager for reporting this ..

The bridge became outdated following yesterday’s mainnet runtime upgrade.
Currently, the bridge is unable to interpret the chain’s events from the latest runtime 149.
Once the operations team upgrades the bridge to the latest version, the bridge will receive all the expired events again and  you should receive the expected amounts promptly on stellar destination.

done https://github.com/threefoldtech/tf_operations/issues/2393

tftBridgeModule.executedBurnTransactions: Option<PalletTftBridgeBurnTransaction>
{
  block: 11,566,905
  amount: 1,000,000,000
  source: 5HW86mnxp4JBpQrASKbHcJPf2Mxmma8CuiB5azZZJq32a38c
  target: GAGUIASHDZZC6LOPKZFCEMU37YSVYV2EAS5TLSE2VGCNOMHIEW2BK6O5
  signatures: [
    {
      signature: gZ1D0JQIcczifb9uuC5+w4lk3Evf4s0OXGgbbGeYHewN12gJQQogO6a8ZC4EW42Fi/gGRwqekyfDlQHpJFd+Cg==
      stellarPubKey: GBNOTAYUMXVO5QDYWYO2SOCOYIJ3XFIP65GKOQN7H65ZZSO6BK4SLWSC
    }
  ]
  sequenceNumber: 167,255,217,736,281,238
}

tftBridgeModule.executedBurnTransactions: Option<PalletTftBridgeBurnTransaction>
{
  block: 11,566,896
  amount: 1,000,000,000
  source: 5HW86mnxp4JBpQrASKbHcJPf2Mxmma8CuiB5azZZJq32a38c
  target: GAGUIASHDZZC6LOPKZFCEMU37YSVYV2EAS5TLSE2VGCNOMHIEW2BK6O5
  signatures: [
    {
      signature: 1qO7gICDgwPe8oHUj8dQSd7PDr+yynsBrxxsbBK20Gns5l8ojZukC86BcIiHBvsdvSjhnZ/muxGdYcL/GrYoBw==
      stellarPubKey: GBNOTAYUMXVO5QDYWYO2SOCOYIJ3XFIP65GKOQN7H65ZZSO6BK4SLWSC
    }
  ]
  sequenceNumber: 167,255,217,736,281,236
}

tftBridgeModule.executedBurnTransactions: Option<PalletTftBridgeBurnTransaction>
{
  block: 11,566,899
  amount: 10,000,000
  source: 5HW86mnxp4JBpQrASKbHcJPf2Mxmma8CuiB5azZZJq32a38c
  target: GAGUIASHDZZC6LOPKZFCEMU37YSVYV2EAS5TLSE2VGCNOMHIEW2BK6O5
  signatures: [
    {
      signature: qbxzxKcTmNXRrBj6XWft9gI8yvrnN9DlotDi9dZGqW9qDiQRKxj6wZohdHIK3okmeZPqyVYcegeelEaS1kKcAA==
      stellarPubKey: GBNOTAYUMXVO5QDYWYO2SOCOYIJ3XFIP65GKOQN7H65ZZSO6BK4SLWSC
    }
  ]
  sequenceNumber: 167,255,217,736,281,237
}

All reported transactions was processed successfully.
Please reopen if needed.

Funds received on the Stellar side. Thanks :+1:


Create GitHub Action to build and validate PR builds

This is tracked internally by task and scheduled for November's sprint.

This isn't currently feasible as we use private artifacts.


Tracking issue for consistent read

In a distributed system, a read request should see the result of a former succeeded write, by wall-clock means, or by pseudo time(log id) means.
To ensure consistency in a distributed system, when a read request is made, it should be able to access the outcome of a previously successful write, either through wall-clock means or by using pseudo time (log id).

[ ] Requirement discussion: #262
[ ] #804
[ ] #805

@drmingdrmer Can I try it?

@HHoflittlefish777
Thank you for your interest in contributing to this feature. However, we are not ready to implement it at this time due to several refactoring tasks that need to be completed first, including updating the leader lease: leader lease is not updated by leader itself, only by followers.
I will contact you when the dependency is ready, and if you are still interested, we can discuss further. Please let me know if you require additional information.
Best regards

@drmingdrmer Thanks for you reply.I am willing to wait until refactor completed.I don't need additional information at the moment. In the period,I would like to try some simple task to familiarize openraft. Is some suitable task I can do?If not, it is no matter.
Wish everything goes well.


UnresolvableObjectException after successful insert (SharePoint)

Hopefully I'm just missing something obvious, but I can't figure out how to convince Simple.OData.Client to give me the object the server returns after an insert. Because of how SharePoint is structured the request is a little strange, and I think it is confusing SOC about what it is getting back:
        async public Task<IContactInfo> AddContact(IContactInfo contact)
        {
            var record = await cc.Client
                .For("SP.List")
                .Key(cc.ContactsId)
                .NavigateTo<SPContactInfo>("items")
                .Set(SPContactInfo.FromContact(contact))
                .InsertEntryAsync();
            return record;
        }

Here is a heavily trimmed version of what SharePoint hands back:
<?xml version="1.0" encoding="utf-8"?>
<entry>
    <id>Web/Lists(guid'a1796420-a6c4-40b3-a4ca-bdf74fc25025')/Items(13774)</id>
    <category term="SP.Data.ContactsListItem" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
        <content type="application/xml">
        <m:properties>
            <d:Id m:type="Edm.Int32">13774</d:Id>
            <d:Fields m:type="Edm.String">Have been omitted</d:Fields>
        </m:properties>
    </content>
</entry>

And the exception
{ "error": {
	"code": "",
	"message": "An error has occurred.",
	"innererror": {
		"message": "Entity type [Lists(%27a1796420-a6c4-40b3-a4ca-bdf74fc25025%27)/Items] not found",
		"type": "Simple.OData.Client.UnresolvableObjectException",
        "stacktrace": "
           at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
           at Simple.OData.Client.BoundClient`1.<InsertEntryAsync>d__15.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
           at Companies360.SPCompanyData.Repositories.SPContactRepository.<AddContact>d__1.MoveNext() in SPContactsRepository.cs:line 28
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
           at Companies360.Api.Controllers.ContactsController.<AddContact>d__3.MoveNext() in ContactsController.cs:line 43
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
           at System.Threading.Tasks.TaskHelpersExtensions.<CastToObject>d__1`1.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
           at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__1.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
           at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__5.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
           at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__15.MoveNext()"
}}}

Intellisense successfully determines the result should be SPContactInfo, but how do I convince SOC?

One more SharePoint problem: On update SP refuses to return the updated record, instead returning a 204 No Content. I know this, but I can't tell SOC, so it blows up with a null reference exception every time, even if resultRequired is false.


Please add support for other databases

I like all your products. I would like to go with the store in production, but your database does not have enough replication. It would be great to see the couchbase or cockrouchdb(postgresql), perhaps through dmbs. Maybe in the form of premium packages?

Hi @ANTEVETY,
thank you, but this is not possible because entire eshop is optimized for NoSQL embedded database and new version will be optimized too. I have had a plan for advanced eshop built on PostgreSQL or MySQL but nobody who should paid for it :-D

If I correctly translated it: "I have had a plan for advanced eshop built on PostgreSQL or MySQL but nobody who should paid for it :-D"... How much money is required to collect in order for an update to support postgres or mongo, couchbase... in the Eshop? Maybe a spoiler on the main page of the site for donations.

I took advantage of free time and wrote on the site a small overview of the main features of the Total. I hope the russian community will somehow help development, and for example it motivates you to supplement and inject to Total products other databases;) Your marketplace works just hidden from passersby... in order to participate I need to become a premium?

Thank you! I really appreciate it. I will think about it, maybe I will a free time on Christmas :-D


should we use d3/src/js ?

Thanks for your great work, I have a small question. What is the difference to use d3.dart with
this way:
import 'package:d3/d3.dart' as d3;
new d3.Selection("body") ...
or this way:
import 'package:d3/src/js/selection.dart' as d3selection;
d3selection.select("boody") ...
the first one is handy and much dart-alike,
but the second one seems much close to the js syntax

You can use either package. The d3/js package is a dynamically typed
wrapper around d3.js. The d3/d3.dart package wraps the d3/js package to
provide type information. The d3/js package might be more useful if you are
translating existing JS code. The d3/d3.dart package is better for
producing idiomatic Dart. However, I don't think the d3/d3.dart package
covers as much of the D3 API.
On 29 October 2016 at 04:04, Eason Pai<EMAIL_ADDRESS>wrote:

Thanks for your great work, I have a small question. What is the
difference to use,
this way:
import 'package:d3/d3.dart' as d3;
new d3.Selection("body") ...
or this way:
import 'package:d3/src/js/selection.dart' as d3selection;
d3selection.select("boody") ...
the first one is handy and much dart-alike,
but the second one seems much close to the js syntax
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rwl/d3.dart/issues/2, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAmLbY6fmNgirwHY3eqfHP2lTtgw--zks5q4re2gaJpZM4KkAiv
.


About unable to generate vectors.txt

Hello, I followed the steps to run the demo.sh file, but vectors.txt was not generated. I would like to ask you if this is normal

It is not normal.  What was the output?  Please use a pastebin or something
like that if there is a lot of output with no obvious errors, or maybe just
post the last few lines


[Question] how to install 1.0.0-beta?

Hi guys,
Sorry for my ignorance, but I tried to "npm install https://github.com/rackt/react-router.git --save-dev" everything went well, but when I start my app, it says:
"ERROR in ./app/js/app.js
Module not found: Error: Cannot resolve module 'react-router' in /Users/goncalvesjoao/projects/lcfe/sandboxes/project/app/js
@ ./app/js/app.js 5:16-39"
webpack can't find react-router module, although I see it in my node_modules folder.
How can I correctly install this new 1.0.0 beta?
Thanks in advance.

npm install react-router@beta --save

Thanks @tmbtech !


SSGCID structures not displayed on gene pages

Reported by Isabelle Phan:
On Thu, Feb 26, 2015 at 3:44 PM, Isabelle Phan <isabelle.phan@seattlebiomed.orgmailto:isabelle.phan@seattlebiomed.org<mailto:isabelle.phan<EMAIL_ADDRESS>wrote:
Hello,
There seems to be an issue with the cross-references from PATRIC to SSGCID. I am preparing our semi-annual report to NIAID and noticed that some PATRIC records are showing erroneous links.
For example,
http://patricbrc.org/portal/portal/patric/Feature?cId=PATRIC.283166.5.NC_005956.CDS.98477.99325.rev&cType=feature
has a link:
Submit a request for structure determination to SSGCIDhttp://patricbrc.org/portal/portal/patric/Feature?cId=PATRIC.283166.5.NC_005956.CDS.98477.99325.rev&cType=feature#
even though that protein has been in our pipeline for years and is cross-linked to PATRIC on our side:
https://apps.sbri.org/SSGCIDTargetStatus/Target/BaheA.00606.a
Would it possible for you to update cross-references to SSGCID on the PATRIC site?
Regards,
Isabelle

Problem was due to change in the ID scheme. Harry fixed the issue. SSGCID structure info and cross links are not available on the gene pages.
https://www.beta.patricbrc.org/portal/portal/patric/Feature?cId=PATRIC.283166.5.NC_005956.CDS.98477.99325.rev&cType=feature


Method to show if element is displayed

Does Puppeteer has API which help detect if element is visible (or displayed), analogue to Selenium (Webdriver) .isDisplayed method?
I found method waitForSelector method which has options visible, but I think if DOM node has display: block; and some parent component has style overflow: hidden and my element outside the parent this waitForSelector returns wrong

Using Intersection Observer would be more appropriate for this case.

@ebidel nice idea, we should use Intersection Observer for the waitForSelector implementation.

Does Puppeteer has API which help detect if element is visible (or displayed), analogue to Selenium (Webdriver) .isDisplayed method?

Not yet, but it makes perfect sense to have ElementHandle.isVisible() that is aligned with waitForSelector's implementation of visibility.

@b-ponomarenko - can you help with a test case where waitForSelector with visible does not work for you?

@yanivefraim yes, simple case is carousel, for example from all carousel item I want choose only visible, but waitForSelector with visible option returns all items.

I try to inject script with IO:
‘’’js
const isDisplayed = await page.evaluate(
() => new Promise(resolve => {
const callback = ([entry]) => resolve(entry.isIntersecting);
const observer = new IntersectionObserver(callback, { root: null, rootMargin: '0px', threshold: 1.0 });
      observer.observe(document.querySelector('.ui-trading-main__product-title'));
      setTimeout(() => observer.unobserve(document.querySelector('.ui-trading-main__product-title')), 0)
    })
);

‘’’
but have error stack trace
‘’’
Error: Protocol error (Runtime.callFunctionOn): Promise was collected undefined
at Promise (/Users/b.ponomarenko/Desktop/puppeteer/lib/Connection.js:202:56)
at new Promise ()
at CDPSession.send (/Users/b.ponomarenko/Desktop/puppeteer/lib/Connection.js:201:12)
at ExecutionContext.evaluateHandle (/Users/b.ponomarenko/Desktop/puppeteer/lib/ExecutionContext.js:79:75)
at ExecutionContext.evaluate (/Users/b.ponomarenko/Desktop/puppeteer/lib/ExecutionContext.js:46:31)
at Frame.evaluate (/Users/b.ponomarenko/Desktop/puppeteer/lib/FrameManager.js:326:20)
at
at process._tickCallback (internal/process/next_tick.js:160:7)
‘’’
what can be wrong?

Hi guys, I am getting bit lost in the various discussions and implementation efforts regarding visibility gong on here :)
You was discussing the visibility in the viewport... I have slightly different problem.
I am trying to wait for the element which is present in the DOM, but is not visible from human point of view - it is overlaid by video.
Is there a way how to detect if the element is visible by human and clickable?
Even if that would be temporary workaround.
If I am posting it into wrong ticket, sorry for that and feel free to correct me.
Thanks!


Tab bar icons support

Hi,
Is there a way to include icons instead of text on each tab?
Thanks.

Sure, just define your TabIcon which will display image and add image source attribute to route (it will be passed to your TabIcon component).

Here is an example of how I did it:
            <View style={styles.tabContainer}>
              <Icon
                name={this.props.tabIcon}
                size={24}
                color={this.props.selected ? '#444' :'#AAA'}
              />
              <Text style={[{color: this.props.selected ? '#444' :'#AAA'}, styles.tabIcon]}>{this.props.title}</Text>
            </View>

I'm using react-native-vector-icons as my icon manager

@jfsemaan feel free to reopen this if you have any other questions


[BUG] Failed upgrade from 1.4.1 to 1.5.1 via helm

Hi, I'm unable to upgrade to the newest longhorn version. Have you got any clue what might be wrong ?
Describe the bug (if you encounter this issue)
Upgrade fails
Error: UPGRADE FAILED: pre-upgrade hooks failed: 1 error occurred:
	* job failed: BackoffLimitExceeded

To Reproduce
 helm upgrade longhorn longhorn/longhorn --namespace longhorn-system

Expected behavior
Upgrades succeeds
Additional context
+ longhorn-pre-upgrade-4djfm › longhorn-pre-upgrade
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade panic: unrecognized command: pre-upgrade
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade goroutine 1 [running]:
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade main.cmdNotFound(0x20adfdb, {0x7ffe1bf170c8, 0x0})
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade 	/go/src/github.com/longhorn/longhorn-manager/main.go:15 +0x67
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade github.com/urfave/cli.ShowCommandHelp(0xc000544f20, {0x7ffe1bf170c8, 0xb})
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade 	/go/src/github.com/longhorn/longhorn-manager/vendor/github.com/urfave/cli/help.go:213 +0x188
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade github.com/urfave/cli.glob..func1(0xc000541040)
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade 	/go/src/github.com/longhorn/longhorn-manager/vendor/github.com/urfave/cli/help.go:21 +0x34
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade github.com/urfave/cli.HandleAction({0x1ccdc60, 0x21aefd0}, 0xb)
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade 	/go/src/github.com/longhorn/longhorn-manager/vendor/github.com/urfave/cli/app.go:523 +0xa8
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade github.com/urfave/cli.(*App).Run(0xc0004a5340, {0xc00004c040, 0x2, 0x2})
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade 	/go/src/github.com/longhorn/longhorn-manager/vendor/github.com/urfave/cli/app.go:285 +0x734
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade main.main()
longhorn-pre-upgrade-4djfm longhorn-pre-upgrade 	/go/src/github.com/longhorn/longhorn-manager/main.go:68 +0xa6a
- longhorn-pre-upgrade-4djfm › longhorn-pre-upgrade

Odd.  "pre-upgrade" is a v1.5 command, and would be appropriate for an upgrade from 1.5.x to something, but it is apparently being given to a 1.4 longhorn-manager which doesn't support it.

That doesnt seem to be a case.
This is the job generated by helm upgrade:
# Source: longhorn/templates/preupgrade-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    "helm.sh/hook": pre-upgrade
    "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation,hook-failed
  name: longhorn-pre-upgrade
  namespace: longhorn-system
  labels:
    app.kubernetes.io/name: longhorn
    helm.sh/chart: longhorn-1.5.1
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/instance: longhorn
    app.kubernetes.io/version: v1.5.1
spec:
  activeDeadlineSeconds: 900
  backoffLimit: 1
  template:
    metadata:
      name: longhorn-pre-upgrade
      labels:
        app.kubernetes.io/name: longhorn
        helm.sh/chart: longhorn-1.5.1
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/instance: longhorn
        app.kubernetes.io/version: v1.5.1
    spec:
      containers:
      - name: longhorn-pre-upgrade
        image: longhornio/longhorn-manager:master-head
        imagePullPolicy: IfNotPresent
        command:
        - longhorn-manager
        - pre-upgrade
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
      restartPolicy: OnFailure
      serviceAccountName: longhorn-service-account

And this is the help of executed command when I run debug pod with the same image as the generated job.
NAME:
   longhorn-manager - Longhorn Manager

USAGE:
   longhorn-manager [global options] command [command options] [arguments...]

VERSION:
   v1.5.0-dev

COMMANDS:
   daemon
   recurring-job
   deploy-driver
   csi
   conversion-webhook
   admission-webhook
   recovery-backend
   post-upgrade
   uninstall
   system-rollout
   migrate-for-pre-070-volumes
   help, h                      Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug, -d     enable debug logging level [$RANCHER_DEBUG]
   --log-json, -j  log in json format [$RANCHER_LOG_JSON]
   --help, -h      show help
   --version, -v   print the version

It really seems to be missing

This image doesn't have pre-upgrade available, which is charts default.
docker pull longhornio/longhorn-manager:master-head

This one does
docker pull longhornio/longhorn-manager:v1.5.x-head

[root@work ~]# docker run  -it  longhornio/longhorn-manager:v1.5.x-head-amd64 /bin/bash
Unable to find image 'longhornio/longhorn-manager:v1.5.x-head-amd64' locally
v1.5.x-head-amd64: Pulling from longhornio/longhorn-manager
ec411b235705: Pull complete
0c1231b4bcaf: Pull complete
a4e1ebc1f445: Pull complete
Digest: sha256:6bf20074ba762b259500a16c7ef8d0e6402aed37b04512066f72c64ab5360b6c
Status: Downloaded newer image for longhornio/longhorn-manager:v1.5.x-head-amd64
0caf8eb0f1f1:/ # lo
loadkeys          local             localectl         login             logname           logsave           look
loadunimap        locale            localedef         loginctl          logout            longhorn-manager  losetup
0caf8eb0f1f1:/ # lo
loadkeys          local             localectl         login             logname           logsave           look
loadunimap        locale            localedef         loginctl          logout            longhorn-manager  losetup
0caf8eb0f1f1:/ # longhorn-manager --help
NAME:
   longhorn-manager - Longhorn Manager

USAGE:
   longhorn-manager [global options] command [command options] [arguments...]

VERSION:
   v1.5.1

COMMANDS:
   daemon
   recurring-job
   deploy-driver
   csi
   pre-upgrade
   post-upgrade
   uninstall
   system-rollout
   migrate-for-pre-070-volumes
   help, h                      Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug, -d     enable debug logging level [$RANCHER_DEBUG]
   --log-json, -j  log in json format [$RANCHER_LOG_JSON]
   --help, -h      show help
   --version, -v   print the version
0caf8eb0f1f1:/ #

Ah, that's the "newest version" referred to.  Despite the title, that would not be an upgrade to 1.5.1, which is tagged, and also in a separate branch.  The docs for 1.5.1 should contain the right recipe for upgrading to it (the current latest release):  https://longhorn.io/docs/1.5.1/deploy/upgrade/
longhorn-manager:master-head is the top of development tree for the next point release, 1.6.  It is intended for development purposes, but may not support all aspects of installation or upgrade yet.

I figured that I was being a fool, as I should of course 'pin' the right versions in my values.


Floating point exception

Dear developers,
I would like to report an error that I get when I try to use HDF5 files in XDS using neggia.
The stacktrace looks like this:
...

 UNTRUSTED_RECTANGLE=   1030.  1041.       0.  3270.   (X1,X2, Y1,Y2)
 UNTRUSTED_RECTANGLE=   2070.  2081.       0.  3270.   (X1,X2, Y1,Y2)
 UNTRUSTED_RECTANGLE=      0.  3111.     514.   552.   (X1,X2, Y1,Y2)
 UNTRUSTED_RECTANGLE=      0.  3111.    1065.  1103.   (X1,X2, Y1,Y2)
 UNTRUSTED_RECTANGLE=      0.  3111.    1616.  1654.   (X1,X2, Y1,Y2)
 UNTRUSTED_RECTANGLE=      0.  3111.    2167.  2205.   (X1,X2, Y1,Y2)
 UNTRUSTED_RECTANGLE=      0.  3111.    2718.  2756.   (X1,X2, Y1,Y2)
 PROCESS #  1 COVERS THE IMAGE RANGE        1 ...      50
 NUMBER OF TRUSTED DETECTOR PIXELS  9486379

 DETERMINATION OF THE DARK CURRENT LOOK-UP TABLE "BLANK.cbf"
 -----------------------------------------------------------

 DARK CURRENT LOOK-UP TABLE IS SET CONSTANT TO        0

 DETERMINATION OF THE GAIN LOOK-UP TABLE "GAIN.cbf"
 --------------------------------------------------
 GAIN LOOK-UP TABLE IS DETERMINED FROM DATA IMAGES       1      50
 AND SAMPLED AT   445 X   467 EQUIDISTENT POINTS ALONG X- AND Y.

 *** DEFINITION OF SYMBOLS ***
  GAIN     = MEAN OF THE GAIN-VALUES ON DATA IMAGE
             "GAIN" RELATES POISSON-DISTRIBUTED X-RAY COUNTS
             TO DATA IMAGE PIXEL VALUES SUCH THAT
             GAIN * X-RAY COUNTS = PIXEL VALUE - DARK CURRENT
 MGAIN     = NUMBER OF SAMPLING POINTS ON DATA IMAGE
             USED FOR DETERMINATION OF GAIN
 I/O-FLAG  = ERROR CODE AFTER ACCESSING DATA IMAGE
              0: NO ERROR
             -1: CANNOT OPEN OR READ IMAGE FILE
             -3: WRONG DATA FORMAT

 FRAME #    GAIN     MGAIN

 [generic_data_plugin] - INFO - generic_open
       + library          = <
 /opt/cecisw/arch/easybuild/2016b/software/Neggia/1.0.1-foss-2016b/lib/dectris-n
 eggia.so>
       + template_name    = <CPS2305_M6_1_data_000001.h5>
 [generic_data_plugin] - ERROR - 'handle' not null
 generic_getfrm: could not open
 /opt/cecisw/arch/easybuild/2016b/software/Neggia/1.0.1-foss-2016b/lib/dectris-n
 eggia.so ier=          -1
 [generic_data_plugin] - INFO - generic_get_header
NEGGIA ERROR: CANNOT READ PIXEL MASK FROM CPS2305_M6_1_data_000001.h5
 nx,ny,nbyte,qx,qy,number_of_frames,info_array(1:5)=           0           0
           0  0.0000000E+00  0.0000000E+00           0           1           0
           5           3          -1
INFO(1:5)=vendor/major version/minor version/patch/timestamp=   1   0   5   3          -1
 generic_getfrm: data are from Dectris
forrtl: severe (71): integer divide by zero
Image              PC                Routine            Line        Source
xds<PHONE_NUMBER>60CF09  Unknown               Unknown  Unknown
xds<PHONE_NUMBER>60B880  Unknown               Unknown  Unknown
xds<PHONE_NUMBER>5BFE52  Unknown               Unknown  Unknown
xds<PHONE_NUMBER>56CB53  Unknown               Unknown  Unknown
xds<PHONE_NUMBER>573951  Unknown               Unknown  Unknown
libpthread.so.0    00007F5A0FE0E370  Unknown               Unknown  Unknown
dectris-neggia.so  00007F5A104F1149  Unknown               Unknown  Unknown
dectris-neggia.so  00007F5A104F1DD8  Unknown               Unknown  Unknown
xds<PHONE_NUMBER>5502FE  generic_getfrm_         24930  MAIN_XDS.f90
xds<PHONE_NUMBER>54D38E  getfrm_                 24993  MAIN_XDS.f90
xds<PHONE_NUMBER>56043B  mkgtbl_                 10733  MAIN_XDS.f90
xds<PHONE_NUMBER>5681BE  init_                    8337  MAIN_XDS.f90
xds<PHONE_NUMBER>408271  xds_                    20918  MAIN_XDS.f90
xds<PHONE_NUMBER>406256  MAIN__                      1  MAIN_XDS.f90
xds<PHONE_NUMBER>4036B6  Unknown               Unknown  Unknown
libc.so.6          00007F5A0F85BB35  Unknown               Unknown  Unknown
xds<PHONE_NUMBER>4035A9  Unknown               Unknown  Unknown

The version corresponding to the tag "v1.0.1" and with the standard CMAKE procedure.
Is it possible that this is coming from your side ?
Thanks in advance !

I've found the error message "NEGGIA ERROR: CANNOT READ PIXEL MASK FROM CPS2305_M6_1_data_000001.h5" in your output. The pixel mask is usually found in the master file (e.g. CPS2305_M6_1_master.h5). Did you process from a data file?

I use the script XDS_to_H5.py, as reported here. The generated XDS.INP is to be found on this gist, but maybe I need to change it.

closing this issue due to inactivity. if problem should still remain please re-open with further information.


Remove unpassable cargo publish check from verify-release-candidate.sh

Resolves https://github.com/apache/arrow-rs/issues/881
The newly added cargo publish checks I added in #856 (see  https://github.com/apache/arrow-rs/pull/856/files#diff-82f32771b6ac9daaadbfb77bd77062d63e131ad18ecd9014ff90eff17865964fR137-R152) do not actually work for any crate other than arrow
See #881 for explanation of why
Basically the issue is that they can't find the correct version of arrow because it hasn't been published

Backporting -- I didn't realize people used the release verification script from the tarball itself. Hopefully this will be in the 6.3.0 release


{Packaging} Use subprocess for better cross-platform support

The entry script az added by #10516 introduced 2 issues on Windows-based Linux, like Git Bash, MinGW, Cygwin.
Quoting issue
Directly calling os.exec will result in the quoting issue mentioned in https://github.com/Azure/azure-cli/pull/13774: spaces are used as argument separator.
According to _exec, _wexec Functions, this is the expected behavior:

Spaces embedded in strings may cause unexpected behavior; for example, passing _exec the string "hi there" will result in the new process getting two arguments, "hi" and "there". If the intent was to have the new process open a file named "hi there", the process would fail. You can avoid this by quoting the string: "\"hi there\"".

subprocess.Popen on the other hand will pre-process the args using the subprocess.list2cmdline function so that args are preserved when they are sent to CreateProcess.
The entry script doesn't wait for azure.cli to finish
When the entry script is executed in Git Bash, it will exit before the actual azure.cli finishes. Note that the prompt $ appears in the wrong location:
user@surface MINGW64 ~
$ az version

user@surface MINGW64 ~
$ {
  "azure-cli": "2.7.0",
  "azure-cli-command-modules-nspkg": "2.0.3",
  "azure-cli-core": "2.7.0",
  "azure-cli-nspkg": "3.0.4",
  "azure-cli-telemetry": "1.0.4",
  "extensions": {
    "application-insights": "0.1.1",
    "interactive": "0.4.4",
    "mesh": "0.10.6",
    "timeseriesinsights": "0.1.2",
    "webapp": "0.2.24"
  }
}

According to _exec, _wexec Functions, os.exec internally uses CreateProcess which doesn't wait for the sub-process.
Using Popen.wait, the entry script can wait for azure.cli to finish before exiting.
BTW, Linux on the other hand doesn't face these 2 issue.

Packaging

I tried this PR and it fixes the problem "The entry script doesn't wait for azure.cli to finish" when I use development mode. Awesome!


Merge pull request #75 from zalando/dev

Merge to_json() changes

Coverage remained the same at 76.161% when pulling 79029a90517092cea8bbc66a10fb1df3609e6762 on master into 667afd02ce82ded60d63ce7127c932d402df318d on dev.

Coverage remained the same at 76.161% when pulling 79029a90517092cea8bbc66a10fb1df3609e6762 on master into 667afd02ce82ded60d63ce7127c932d402df318d on dev.

Coverage remained the same at 76.161% when pulling 79029a90517092cea8bbc66a10fb1df3609e6762 on master into 667afd02ce82ded60d63ce7127c932d402df318d on dev.


Adding compatibility for modded trees through datapack?

Make sure you are not opening a duplicate.

[X] I have searched the issue tracker and did not find an issue describing my bug/feature.

Description of your suggestion.
Others have mentioned that this mod only works with vanilla log variants. I would like to know if there is a way to add compatibility with modded logs through a datapack? If so, could you maybe provide a guide or template
I would love to use this mod but it just doesn't work with biome mods that add new trees

I agree

Me too. Please do this as soon as possible.


Is it planned to train Phoenix-13B?

Thank you for releasing an excellent chatbot model!
I think this is one of the most fluent and intelligent chatbots in my language.
By the way, is it planned to train and release Phoenix-13B?
Thank you in advance!

Hi @Itto1992,
Thanks for your attention and your kind words! :-)
We are planning to release larger models. Many efforts are needed, e.g., training a backbone (since no larger-size BLOOM backbones exist except for 175b).
Please stay tuned!
Best,
Zhihong

I really appreciate your prompt response.
I'm looking forward to the release:)


Mobile responsivity issues

Can be seen on: mobile 360px width
transfer:

burguer menu spacing before icon and shopping cart alignment:

maybe we could split the sender into 2 lines like this? since the qty length is not fixed and it did not display well in such small width
wdyt? @exezbcz

@floyd-li valid, thank you


Add scss pre-processor to webpack

Some of this was merged in https://github.com/Capgemini/react-scaffold/commit/710a0b8321ee1b0be3ce2c070c4e05fd2bd36ba0
We still need to implement the styles decorator


Duplicated

When I extract metadata from the 'studies.json' I get two with different descriptions:
These studies should then have different names and ids...
[
    {'id': 1, 'name': 'Dutch elderly', 'description': 'This data was measured in 1982 in cooperation with Gemeentelijke Dienst Verpleging en Verzorging, the association for elderly care and housing in The Hague.', 'created_at': '2020-04-15T12: 19: 12+00: 00', 'updated_at': '2020-04-15T12: 25: 12+00: 00', 'sort_position': 7, 'code': 'gdvv1984', 'publication_date': '1984', 'measurement_date': '', 'sources': '', 'people': '- J.F.M. Molenbroek\n- J.J. Houtkamp\n- A.K.C. Burger', 'publications': '- Anthropometry of elderly people in the Netherlands; research and applications by JFM Molenbroek. Applied Ergonomics, volume 18 (1987), issue 3, page 187-199\n- IO bijzondere onderwerpen 16, bejaardenanthropometrie by JFM Molenbroek, JJ Houtkamp and AKC Burger. Technische Hogeschool Delft,
        1983', 'url': '', 'has_shape_data': False, 'isLocked': 1, 'min_age': 53, 'max_age': 106, 'individuals_count': 822, 'age_groups': [
            {'id': 48, 'sort_position': None, 'min_age': 53, 'max_age': None, 'study_id': 1, 'name': '53+'
            }
        ], 'measures': [
            1,
            2,
            3,
            4,
            6,
            9,
            13,
            14,
            16,
            17,
            18,
            21,
            22,
            24,
            29,
            30,
            32,
            43,
            47,
            56,
            75,
            76,
            77,
            78,
            79,
            80
        ]
    },
    {'id': 3, 'name': 'Dutch elderly', 'description': 'The TU Delft Geron project was a national study in which 750 subjects, who lived independently, were assessed. In total about 80 variables, all more or less important for product use, were measured. The sample consisted of four age groups ranging from 50 to over 80 years of age; a group of young people (20 - 30 years) was also studied for the purpose of comparison (see DINED 2004 table). Women and men participated in about equal numbers.', 'created_at': '2020-04-15T12: 20: 06+00: 00', 'updated_at': '2020-04-15T12: 25: 10+00: 00', 'sort_position': 5, 'code': 'geron1998', 'publication_date': '1998', 'measurement_date': '', 'sources': '', 'people': '- C.E.M. van Beijsterveldt\n- J.M.Dirken (Project leader)\n- J.J.Houtkamp\n- J.F.M.Molenbroek\n- L.P.A. Steenbekkers\n- A.I.M.Voorbij', 'publications': '- L.P.A. Steenbekkers, C.E.M. van Beijsterveldt (eds), J.M.Dirken, J.F.M.Molenbroek, A.I.M. Voorbij and J.J. Houtkamp Design-relevant characteristics of ageing users, Delft University Press 1998 (per 1-1-2006 IO-Press Amsterdam)', 'url': '', 'has_shape_data': False, 'isLocked': 1, 'min_age': 50, 'max_age': 94, 'individuals_count': 627, 'age_groups':
    [
            {'id': 9, 'sort_position': None, 'min_age': 50, 'max_age': 54, 'study_id': 3, 'name': '50–54'
            },
            {'id': 10, 'sort_position': None, 'min_age': 55, 'max_age': 59, 'study_id': 3, 'name': '55–59'
            },
            {'id': 11, 'sort_position': None, 'min_age': 60, 'max_age': 64, 'study_id': 3, 'name': '60–64'
            },
            {'id': 12, 'sort_position': None, 'min_age': 65, 'max_age': 69, 'study_id': 3, 'name': '65–69'
            },
            {'id': 13, 'sort_position': None, 'min_age': 70, 'max_age': 74, 'study_id': 3, 'name': '70–74'
            },
            {'id': 14, 'sort_position': None, 'min_age': 75, 'max_age': 79, 'study_id': 3, 'name': '75–79'
            },
            {'id': 15, 'sort_position': None, 'min_age': 80, 'max_age': None, 'study_id': 3, 'name': '80+'
            }
        ], 'measures': [
            2,
            3,
            4,
            5,
            6,
            9,
            12,
            13,
            14,
            15,
            16,
            17,
            19,
            20,
            21,
            22,
            25,
            30,
            31,
            32,
            33,
            41,
            42,
            43,
            44,
            45,
            46,
            54,
            55,
            56,
            57,
            58,
            59,
            60,
            61,
            62,
            63,
            64,
            65,
            66,
            67,
            68,
            69,
            70,
            71,
            72,
            73,
            74,
            80
        ]
    }
]

The problem is not that the studies are duplicated, the names are duplicated. But there is another related bug, when I query from study number 6 until study 11, I get no data from the mysql database. Also study 12 returns no data.
As a result we have less studies in files than the ones we can actually count and appear in the studies.json metadata.

This might be related to the studies that have no measurements attached but only have summary statistics stored in the database. The 1D database tool shows all studies. The Ellipse tool shows only those studies for which we have measurements for the individual, but not those which only have summary statistics.
I'm not sure how this differences is exposed in the API, if it even is..

Thanks for the clarification @toonhuysmans maybe we can address that in the next catch up.


Create IdpProxies for NodeJS

Some of the APIs used are not supported by the nodejs and should be replaced by something else

@vitormsilva I have pushed a temporary version of the Nodejs IdP Proxy for Google's IdP. Once IdP Proxies can be deployed in the nodejs environment I'll improve on this IdP Proxy.
Proxy here

@pchainho Now that I can successfully deploy IdP Proxies on Nodejs, I'll be improving on the temporary version of the google IdP Proxy for Nodejs.

I have a working IdP Proxy for the google IdP. Some changes are still needed to have nodejs mutual authentication. I'll update on that here. For now I'll close this comment and I'll leave this issue open (dev-runtime-core#149 - MutualAuthentication when we using nodejs environemnt).


Events in all supported languages

Issue in ./help/admin/admin/logs.md
The page shows only the Event Names in English. However if people use Adobe Analytics in another language, the logs are in that particular language. Please add a list of all events in all language versions so we don't need to waste days to figure out all variants...

@loldenburg - you can access the language versions by clicking the globe icon at the top right of the Help page, then selecting the language in which you wish to view the events.

@MicaPete that is not what I meant. That is an autotranslation which means it is terrible to read, and what we really need is to make sense of the multitude of languages in the Logs. So even for English, the Event Names do not reflect what we get in the logs. There needs to be a list of the exact wording. Otherwise we spend weeks just trying to figure out stuff like this:
The same Events logged in many different languages... An absolute horror to do any analysis on. So if you want people to use the logs for something useful, you need to provide proper documentation.

@gigazelle - can you take a look at this?

Hi Lukas, this is an excellent point that you bring up. Meike and I will talk with PM/engineering to get their take on making logs language-agnostic. The outcome I hope to achieve is a product update where events are recorded the same regardless of language. If the powers that be determine that this is WAD, then we will get each event documented in each language.

Hi Lukas, we finished the meeting today. We are tracking this internally via AN-284631. Feel free to reach out to me on Slack if you'd like to check in on it. Thanks!


speak_streamed on google issue for using mp3

Prerequisites

[x] Can you reproduce the problem?
[x] Are you running the latest version?
[x] Did you check the documentation?
[x] Did you perform a cursory search?

For more information, see the contributing guide.
Description
Function speak_streamed can be configured to use mp3. However since google tts does not support mp3 yet, it will produce error
Steps to Reproduce

call speak_streamed like this
tts.speak_streamed("test google", str(output_file), audio_format='mp3')

Expected behavior: [What you expected to happen]

playback will produce error, and no sound came out
mp3 will be created

Actual behavior: [What actually happened]
mp3 is not implemented for playback, only for saved to file

I need to test this
import sounddevice as sd
import numpy as np
from io import BytesIO
import threading

class GoogleTTS(AbstractTTS):
    # Your class setup and init methods remain unchanged

    def speak_streamed(
        self,
        text: str,
        save_to_file_path: Optional[str] = None,
        audio_format: Optional[str] = "wav",
    ) -> None:
        """
        Synthesize text and stream it for playback using sounddevice.
        Optionally save the audio to a file after playback completes.

        :param text: The text to synthesize and stream.
        :param save_to_file_path: Path to save the audio file (optional).
        :param audio_format: Audio format to save (e.g., 'wav', 'mp3', 'flac').
        """
        # Synthesize audio to bytes
        audio_bytes = self.synth_to_bytes(text)

        if audio_format == "mp3":
            # Decode MP3 to PCM
            pcm_data = self._convert_mp3_to_pcm(audio_bytes)
            self.audio_rate = 22050  # Or set based on your TTS output rate
            channels = 1  # Assuming mono output
        else:
            # Directly use PCM data for other formats
            pcm_data = audio_bytes
            channels = 1

        # Playback in a new thread for non-blocking audio
        threading.Thread(target=self._play_pcm_stream, args=(pcm_data, channels)).start()

        # Optionally save to file
        if save_to_file_path:
            with open(save_to_file_path, "wb") as f:
                f.write(audio_bytes)

    def _play_pcm_stream(self, pcm_data: bytes, channels: int):
        """Streams PCM data using sounddevice."""
        audio_data = np.frombuffer(pcm_data, dtype=np.int16).reshape(-1, channels)
        with sd.OutputStream(
            samplerate=self.audio_rate,
            channels=channels,
            dtype="int16",
        ) as stream:
            stream.write(audio_data)

@Chafid  - this is largely right - BUT im lost. Shouldnt this largely be dealt by abstract tts rather than in the engine? I note a lot of missing stuff about format handling but im thinking abstract should be dealing with this


[Feature] Carousel on homepage scrolling through testimonials from customers.

Is your feature request related to a problem? Please describe.
Wanted to spiff up the home page, so I wanted to start a carousel type widget that I could pass specific frontmatter to to have it carousel testimonials and cool facts about the project.
Describe the solution you'd like
Something that goes into the Jekyll _includes folder specifically designed to create a classic "carousel" widget on the screen. Based on the frontmatter content, we can either display an interesting / eyebrow raising fact about the project (Did you know we've had X contributors outside of AWS this month?) or display a testimonial. The widget will auto scroll through the available content it has at regular intervals.
Describe alternatives you've considered
There's a carousel widget for jekyll available as a non plugin, but unfortunately it only supports displaying images at certain ratios. Facts and testimonials aren't always the same size. We need a carousel that renders html in its panels.
Additional context
I've experimented adding it to the top of the homepage, but as I've mentioned before, not all quotes are the same length, so as they move through I can't quite keep it from pushing the rest of the page down.

Something like this: https://github.com/nateynateynate/project-website/tree/1034-carousel


A new implementation for OCI/OKE (Oracle cloud)

Is an existing page relevant?
No response
What karpenter features are relevant?
Hello guys.
I'm considering writing a new implementation for OCI/OKE (due the fact the we've moved to Oracle) and I found that the main.go contains specific AWS code: https://github.com/aws/karpenter/blob/main/cmd/controller/main.go#L27
So my question is - what is the best way to add a new implementation? I can't find any documentation about it.
How should the docs be improved?
Please add a new section 'How to add a new implementation for CLOUD_NAME

We'd love to have more providers implemented for Karpenter! A new provider would live in its own Git repo, not within the github.com/aws/karpenter repo. An example implementation of another provider can be found here: https://github.com/bwagner5/karpenter-k3d

@bwagner5
Thanks, will take a look!

@bwagner5
I've found that some of flags are specific to AWS:
	// AWS Specific
	f.StringVar(&opts.ClusterName, "cluster-name", env.WithDefaultString("CLUSTER_NAME", ""), "The kubernetes cluster name for resource discovery")
	f.StringVar(&opts.ClusterEndpoint, "cluster-endpoint", env.WithDefaultString("CLUSTER_ENDPOINT", ""), "The external kubernetes cluster endpoint for new nodes to connect with")

But they are validated by default.
func (o Options) Validate() (err error) {
	err = multierr.Append(err, o.validateEndpoint())
	if o.ClusterName == "" {
		err = multierr.Append(err, fmt.Errorf("CLUSTER_NAME is required"))
	}

Is it expected?

We haven't quite untangled are flags yet, but we should definitely remove the dependency on those.

See: https://github.com/aws/karpenter/issues/936

Closing since we will now be tracking these requests in the mega issue #2532


GitHub auth doesn't work

geode index login creates the code and github successfully authorizes it

but after that, cli keeps saying "Checking login status" making you unable to login

could be a server bug, but will have to check

issue still exists pls fix thanks


Add xrd file readers from xrdpattern

Many popular file formats from xrd instruments are supported by aimat-lab/xrdpattern. Test if they can be used in nomad_measurement.xrd.
Here's the link to the repo: https://github.com/aimat-lab/xrdpattern

Together with Stepanka @lankovas and @RubelMozumder, we found that bruker raw reader from xrdpattern only extracts the x-y data along with a scant amount of metadata. We need to explore if this is the limitation of the reader (in which case we might want to extend this reader) or if the bruker raw file doesn't contain much metadata.
Remark: xrdpattern seems to be a Python wrapper around the C++ based library xylib


Add --no-color option

Make the --color=false option, work properly.
The --no-color feature appears to be not be working properly, I have more work to do here. But this is a simplified fix.
Interestingly enough, this feature is in the documentation. (My guess, is maybe at one time, we had this --no-color feature?)
$ foreman --help start
Usage:
  foreman start [PROCESS]

Options:
  -c, [--color], [--no-color]          # Force color to be enabled
  -e, [--env=ENV]                      # Specify an environment file to load, defaults to .env
  -m, [--formation="alpha=5,bar=3"]    # Specify what processes will run and how many. Default: "all=1"
  -p, [--port=N]
  -t, [--timeout=N]                    # Specify the amount of time (in seconds) processes have to shutdown gracefully before receiving a SIGKILL, defaults to 5.
      [--timestamp], [--no-timestamp]  # Include timestamp in output
                                       # Default: true
  -f, [--procfile=PROCFILE]            # Default: Procfile
  -d, [--root=ROOT]                    # Default: Procfile directory

Start the application (or a specific PROCESS)

Here is a pic of my terminal. it is hard to talk colors without some visuals

Thor already supports the --no variants to boolean options. You could just check it for false (will be nil if unspecified)

Ah, I see, I wondered how that --no-color option was getting into the docs. It was a Thor thing.
Thanks for the pointers, I will work towards your suggested approach. And add tests of course.

I've updated things. However, adding tests in rspec that involve the def color? method, which relies on isatty, was getting nutty.
So I did manual testing and left it at that.


[code-coverage] update admin and frontend client to use generated code

What changed?
frontend client wrapper is generated using gowrap as in #5701
Will have an follow up PR for matching client

Why?
To improve code coverage
To reduce manual work when new methods are added in the future

How did you test it?
unit tests

Potential risks

Release notes

Documentation Changes

Pull Request Test Coverage Report for Build 018e09e9-4eac-427a-9f7f-d7dbd35a9713
Details

6 of 6 (100.0%)  changed or added relevant lines in 2 files are covered.
44 unchanged lines in 10 files lost coverage.
Overall coverage increased (+0.2%) to 63.106%

Files with Coverage Reduction
New Missed Lines
%

common/task/weighted_round_robin_task_scheduler.go
1
89.05%

service/history/execution/mutable_state_task_refresher.go
1
68.99%

common/task/fifo_task_scheduler.go
2
87.63%

common/task/parallel_task_processor.go
2
93.06%

service/history/execution/mutable_state_util.go
2
37.63%

service/history/task/transfer_standby_task_executor.go
2
86.6%

service/matching/matcher.go
2
90.72%

service/history/task/fetcher.go
4
86.08%

common/persistence/sql/workflowStateMaps.go
8
81.25%

service/history/task/task_util.go
20
70.57%

Totals

Change from base Build 018dfbb1-48ef-4faa-92f4-b269d8ca713f:
0.2%

Covered Lines:
92974

Relevant Lines:
147330

💛  - Coveralls


Captcha verification is causing issues

While fetching the answers, sometimes Chegg redirects to Recaptcha verification. This will leads to script failure. Please help with this issue.

You need to limit your requests, we won't be working to bypass the captcha

You need to limit your requests, we won't be working to bypass the captcha

How many requests in a time period do you believe would cause the captcha?


restart jupyter kernel only works through clicking restart button in interactive window

Environment data
VS Code version: 1.51.1
Extension version (available under the Extensions sidebar): 2020.11.371526539
Jupyter Extension version (available under the Extensions sidebar): v2020.11.399280825
OS (Windows | Mac | Linux distro) and version: Mac Pro, mac Catalina 10.15.7
Python and/or Anaconda version: 3.7
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): Conda virtual environment
Jupyter server running: Local
Wrong behaviour
The jupyter restart kernel keyboard shortcut does not work, even when triggering it with the command pallete. It does work when i click the restart icon in the interactive window. How could i trigger the restart kernel again with a keyboard shortcut? I already resetted all my keyboard shortcuts so that could not be the issue.

Thanks for the bug.
Restarting the kernel in the interactive window requires the window has focus.

Hi @rchiodo
Thanks for you reply, but i actually mean the keyboard shortcut when coding in the .py file, like in a jupyter cell with #%%. Also, every time when i restart the kernel it asks as an option: yes, and don't ask again. But when i click the yes don't ask again it just keep asking me that question when I restart the kernel later on, it does not remember my choice.

But when i click the yes don't ask again it just keep asking me that question when I restart the kernel later on, it does not remember my choice.

That should have unchecked this setting here:

If it didn't you should be able to uncheck it yourself.
Keyboard shortcut while a python file has focus does make sense. We should change the code to make that possible too.

That should have unchecked this setting here

Where could i find that setting?

Keyboard shortcut while a python file has focus does make sense. We should change the code to make that possible too.

Thanks for considering it, it was a functionality that I used a lot when it was possible with the keyboard shortcuts originally falling under the datascience name instead of the jupyter name.

Where could i find that setting?

Command Palette: Open Settings UI and type 'kernel restart' should find it.

Command Palette: Open Settings UI and type 'kernel restart' should find it.

Thanks i found it, however, I get the error that i am not able to write into users settings. my user settings are:
{
    "terminal.integrated.inheritEnv": false,
    "editor.suggestSelection": "first",
    "vsintellicode.modify.editor.suggestSelection": "automaticallyOverrodeDefaultValue",
    "window.zoomLevel": 0,
    "editor.minimap.enabled": false,
    "python.showStartPage": false,
    "python.languageServer": "Microsoft",
    "workbench.colorTheme": "Cobalt2",
    "editor.fontSize": 15,
    "python.autoComplete.extraPaths": [
        '/Users/name/anaconda3/envs'
    ],
    "jupyter.sendSelectionToInteractiveWindow": true
}

I get the error that i am not able to write into users settings.

Not sure why you can't write to that file, but that's the reason why the message keeps popping up.

I will look online to find a solution, thanks for your help!

Closing in favor of https://github.com/microsoft/vscode-jupyter/issues/6996


Consider adopting prettier

https://github.com/prettier/prettier has a lot more details.
It's been adopted by quite a few projects as the de facto style guide (ie. run your code through it and call it a day - similar to gofmt).
With a few configuration options
prettier --trailing-comma=es5 --single-quote --no-bracket-spacing

the output comes pretty close to our ESLint rules.
The biggest offender I've found so far is we prefer
(arg) => { /* ... */ }

over
arg => { /* ... */ }

for lambdas that take a single argument.

Yeah, I would love to recommend this, I'm willing to relent on the single param functions (even though it's a terrible choice). We had been waiting on that rule, and on full TypeScript support, before jumping on it, and now that TS is in it seems reasonable. What do you think we'd need to do here to make this a clear recommendation? Instructions on how to use it with the linter? Recommended options? An examples use of it in core?

What do you think we'd need to do here to make this a clear recommendation? Instructions on how to use it with the linter? Recommended options? An examples use of it in core?

As far as I can tell, most projects that have adopted it go through a process along the lines of

Update ESLint rules to be compatible with Prettier
Add a yarn run prettier that formats everything with the preferred options
Convert the entire code base in one fell swoop
Implement something like https://github.com/facebook/react/pull/9187 using prettier --list-different on CI to prevent any un-prettier-ed code from being committed

^ once that's done people can update their editor configs or pre-commit hooks to suit their workflows.

As discussed with @lemonmade over Slack, I'd like to do a spike on this and see how far I go towards seamless integration of prettier in our current process. As someone who's just joined I feel like this could be super useful to other new developers so worth investigating.
cc @dfmcphee

+1 for this, I think this would be a great leap towards a more consistent codebase, especially if implemented at the CI level and as a pre-commit hook.
As far as pre-commit hooks go, a lot of large-scale projects nowadays are using a combination of husky and lint-staged to make them easily shareable across the team without needing to be manually installed and to make sure to only run against staged files so they run faster. You can check out the babel package.json for an example of how that works in practice.

Thanks for the pointers. I was initially thinking we could encourage developers to install a Prettier extension in their text editor, but I guess pre-commit hooks scale much better!
I haven't had experience with modifying code at the CI level. Would that be something developers are comfortable with?

Please check with #dev-accel before adding prehooks.  Last time I discussed that, they had a strong preference for finding alternatives.

That's good to know! I'll make sure to have this in mind and will chat in case hooks seem like the best solution.

Closing as we are now leveraging prettier!


Reading output_filename.npz

I tried to produce the file 'output_filename.npz' using the command:
tape-embed transformer my_input.fasta output_filename.npz bert-base --tokenizer iupac.
The file is correctly created, but when I try to import it, I can't access the data correctly.
First (following the instruction reported in README file), using the comand:
arrays = np.load('output_filename.npz', allow_pickle=True),
the file is correctly loaded. Then, I created a list of both the keys (in my case the protein sequences id) and the values (the dictionaries of the 'pooled' and 'avg' vectors associated with the protein id) :
keys = list(arrays.keys())
values = []
for key in keys:
values.append(arrays[key])
But the values of each key in my case are numpy.arrays of size 0, and not dictionaries as indicated in the README file. I can print it, but I don't have the possibility to access the 'pooled' and 'avg' fields as I would like. What am I doing wrong?
Below are some prints of what I mentioned above:

Hey!, I was stuck at the same but if you use arrays[key].item(). This should work.

Ah yes - when saved as a numpy file, numpy makes everything into a numpy array. So dictionaries get turned into numpy object arrays. Calling .item() should give you the result.


Service monitor relabelings

Description
The PR allows to change service monitor relabeling configuration.
Testing
Run helm template.
Checklist

[x] No need to update CHANGELOG.md.
[x] This is unlikely to impact how Ambassador performs at scale.
[x] My change is adequately tested.
[x] No need to update DEVELOPING.md

[x] No need to update CHANGELOG.md.

Actually, this should probably be mentioned as a feature in charts/emissary-ingress/CHANGELOG.md.

Broadly: This change looks good. I'd appreciate if you could change the indentation and update the CHANGELOG, but if not: no worries, a maintainer can do that.


ECFP Attribution Radius

Would like to exclude ECFP substructures with 1 or 2 atoms, since they are not that useful.

Also, add feature to use fixed length ECFP vectors.

Updated title to reflect that we decide radius was fine


Adding my information to GSoC-interest.md

Please @quirrone squash the commits and change the commit message to:
add information about @quirrone

And very likely, rebase it with the current master, since it seems to have some conflicts.


Vertically align text in tabs with rest of status bar

Before

After

The two screenshots look identical :sweat_smile:

I have to expand the images to find it, but there is a very small difference.

It may be more noticeable on retina displays.


Refactored example to make it work on windows

The windows implementation of the Daemon interface method Run now
uses golang.org/x/sys/windows/svc/debug.Run to run the service in
interactive mode. This means that the Run method of Executable
interface is never invoked on windows.
Updated golang.org/x/sys dependency to latest version.
Some cleanup in daemon_windows.go to make linter happy.

Closing PR because it was done against wrong branch (master instead of develop)


Fix/cleanup datastore elasticsearch impls

Should do thorough testing of this-- it is unclear why we were still using the non-Kotlin version.

Overriding and merging with 1 approval.


Possible EventEmitter memory leak detected

I have set the maxConcurrency to 2, and ran a load test with 100 requests in parallel. But the puppeteer-cluster launched 100 workers and crashed.
I believe the library should queue the requests and should not open 100 pages to handle Concurrency and must not crash. Here is the code
Launcher
const puppeteer = require('puppeteer-extra');
const { Cluster } = require('puppeteer-cluster');

const getCluster = async () => {
    const cluster = await Cluster.launch({
        concurrency: Cluster.CONCURRENCY_CONTEXT,
        maxConcurrency: 2,
        monitor: false,
        puppeteer,
        puppeteerOptions: {
            executablePath: process.env.CHROMIUM_PATH || null,
            args: ['--no-sandbox', '--disable-setuid-sandbox']
        }
    });
    return cluster;
}

module.exports = getCluster;

Router
var express = require('express');
var router = express.Router();
const getCluster = require('../services/cluster');

// setup server
router.get('/', async function (req, res) {
    try {
        const cluster = await getCluster();
        await cluster.queue(req.query.url, async ({ page, data : url }) => {
            let response = await page.goto(url);
            const title = await page.title();
            console.log(`${url} - ${response.status()}`);
            res.end(title, 'application/json');
        });

    } catch (error) {
        if (!res.headersSent) {
            res.status(400).send(error.message);
        }
    }
});

module.exports = router;

Error screenshot

Task manager

Closing this. I was mistaken and using the getCluster() inside router. While it should be outside as static variable to re-use.

Thanks for reporting back.


Detailed explaination about the patches for Ubuntu.

As per this issue, which says that[emphasis mine]:

With all kernels v.416+ you get the standard HW timestamp for free (w/o kernel patches), You'd still need to apply the patches if you need the extended frame metadata - exposure, gain, laser poser, etc. But if you're only interested in getting HW timestamps then using unmodified kernel should suffice.

Could somebody give some detailed explanation about the pitches for Ubuntu?
What modules do the aforementioned pitches have modified?

Hi @sunlong169  Notes about the changes made by the patching process can be found within the patch script patch-realsense-ubuntu-lts.sh at the link below.  The script makes changes to the uvcvideo and videobuf2 modules.
https://github.com/IntelRealSense/librealsense/blob/master/scripts/patch-realsense-ubuntu-lts.sh

Hi @sunlong169  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.


New feature: Blacklist (or whitelist) type with file inclusion

For better use of blacklist and withelist type shuld be possible to use file lists. When the blackist contais a lot of fileds drill down the file may be not easy, it's better to use a file separate form the rule file and include the list of fileds with inlude option for example o similar.
ex.
`include:

blackilist_file.yaml`

@chicco27
I would like to apply a method to whitelist IP address and specific string. Currently, I have a cardinality rule and I would like to do whitelist on few strings and IP address by combining the rule as shown below. The rule runs without any error but it's not working as expected. Can someone assist me is this a right method to combine the rule for whitelist?
Main Rule
type: cardinality
filter:
 - terms:
     rule_name:
         -  whitelist-zone  --> Rule name for whitelisting zone
         -  whitelisti-ip  --> Rule name for whitelisting IP address

whitelist-zone
root@ubuntu:/tmp/elastalert# more whitelist-zone.yaml
# ElastAlert Rule Name

name: whitelist-zone

# ElasticSearch host and port details

es_host: <IP_ADDRESS>

es_port: 9200

# ElasticSearch Index Name

index: graylog_0

# Timestamp added to overcome ES graylog parsing issue

timestamp_field: timestamp
timestamp_type: custom
timestamp_format: '%Y-%m-%d %H:%M:%S.%f'
timestamp_format_expr: 'ts[:23] + ts[26:]'

doc_type: message

#writeback_index: elastalert_status

# Network Port scan aggregation Rule 50 counts in 60 Seconds

type: whitelist

compare_key: srx-source-zone-name

whitelist:
 - "!file /opt/whitelist-zone.txt"

whitelisti-ip

root@ubuntu:/tmp/elastalert# more whitelist-ip.yaml
# ElastAlert Rule Name

name: whitelisti-ip

# ElasticSearch host and port details

es_host: <IP_ADDRESS>

es_port: 9200

# ElasticSearch Index Name

index: graylog_0

# Timestamp added to overcome ES graylog parsing issue

timestamp_field: timestamp
timestamp_type: custom
timestamp_format: '%Y-%m-%d %H:%M:%S.%f'
timestamp_format_expr: 'ts[:23] + ts[26:]'

doc_type: message

#writeback_index: elastalert_status

# Network Port scan aggregation Rule 50 counts in 60 Seconds

type: whitelist

compare_key: srx-source-address

whitelist:
  - "!file /opt/whitelist-IP.txt"

Files

root@ubuntu:/tmp/elastalert# cat /opt/whitelist-IP.txt
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
root@ubuntu:/tmp/elastalert# cat /opt/whitelist-zone.txt
EXTERNAL
Internet
root@ubuntu:/tmp/elastalert#

http://elastalert.readthedocs.io/en/latest/ruletypes.html?highlight=import#import

@Qmando
Thanks!!!
Please confirm the below mentioned format is correct ?
import:
Rule name of whitelist
- whitelist-ip.yaml
- whitelist-zone.yaml

In some rule.yaml
name: blah blah
import: whitelist-stuff.yaml

in whitelist.yaml
whitelist:
 - "<IP_ADDRESS>"
 - "<IP_ADDRESS>"

You can only import one file at the moment. Though, that file can also import things.
The imported file will also overwrite any options in the first rule, so you can't have two whitelists and make them merge together.

@Qmando
Apologize for annoy persistently, I have followed the above-mentioned steps, but I'm not able to obtain my requirements.
Please find the steps below

I have created a rule name  test-srx.yaml  and specified the import: whitelist-zone.yam within the rule.

Main rule test-srx.yaml
root@ubuntu:/tmp/elastalert# more test-srx.yaml

name: SRX Port Scan Detection

es_host: <IP_ADDRESS>

es_port: 9200

index: graylog_0

timestamp_field: timestamp
timestamp_type: custom
timestamp_format: '%Y-%m-%d %H:%M:%S.%f'
timestamp_format_expr: 'ts[:23] + ts[26:]'

doc_type: message

type: cardinality

query_key: [srx-source-address, srx-destination-address]
cardinality_field: srx-destination-port
max_cardinality: 50

timeframe:

    seconds: 60
filter:
 - query_string:
      query: "srx-rt-flow: RT_FLOW_SESSION_DENY"

import: whitelist-zone.yaml

aggregation:

   minutes: 2

whitelist-zone.yaml

root@ubuntu:/tmp/elastalert# more whitelist-zone.yaml

name: SRX Port Scan Detection

es_host: <IP_ADDRESS>

es_port: 9200

index: graylog_0

timestamp_field: timestamp
timestamp_type: custom
timestamp_format: '%Y-%m-%d %H:%M:%S.%f'
timestamp_format_expr: 'ts[:23] + ts[26:]'

doc_type: message

type: whitelist

compare_key: srx-source-zone-name

ignore_null: true

whitelist:

 - "EXTERNAL"
 - "Internet"

config.yaml

#rules_folder: example_rules
rules_folder: Network_scan

root@ubuntu:/tmp/elastalert# ls -ls Network_scan
total 8
4 -rw-r--r-- 1 root root 2479 Jul 15 10:38 test-srx.yaml
4 -rw-r--r-- 1 root root  504 Jul 15 10:38 whitelist-zone.yaml

Running config.yaml  Produces the below error message.

root@ubuntu:/tmp/elastalert# python -m elastalert.elastalert  --start NOW --verbose --config config.yaml
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/elastalert/elastalert/elastalert.py", line 1730, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/tmp/elastalert/elastalert/elastalert.py", line 1724, in main
    client = ElastAlerter(args)
  File "/tmp/elastalert/elastalert/elastalert.py", line 111, in __init__
    self.conf = load_rules(self.args)
  File "elastalert/config.py", line 457, in load_rules
    raise EAException('Error loading file %s: %s' % (rule_file, e))
elastalert.util.EAException: Error loading file Network_scan/whitelist-zone.yaml: Invalid Rule file: Network_scan/whitelist-zone.yaml
{'whitelist': ['EXTERNAL', 'Internet'], 'rule_file': 'Network_scan/whitelist-zone.yaml'} is valid under each of {'properties': {'type': {'pattern': '[.]'}}, 'title': 'Custom Rule from Module'}, {'properties': {'type': {'enum': ['any']}}, 'title': 'Any'}

Failed validating 'oneOf' in schema:
    {'$schema': 'http://json-schema.org/draft-04/schema#',
     'definitions': {'arrayOfStrings': {'items': {'type': 'string'},
                                        'type': ['string', 'array']},
                     'arrayOfStringsOrOtherArrays': {'items': {'type': ['string',
                                                                        'array']},
                                                     'type': ['string',
                                                              'array']},

@Qmando
Need your assistance on the above issue  #572

@Qmando
I tried a different combination but still no luck. Not sure what I'm missing here.
As per the document Ifimport: whitelist-zone.yamlspecified it includes all the settings from this yaml file. This allows common config options to be shared.
Note that imported files that aren’t complete rules should not have a .yml or .yaml suffix so that ElastAlert doesn’t treat them as rules. I have tried to remove the.yamlextension and tried but still, the rule is not whitelisting the specified string.
I have attached the rule for reference.
test-srx.zip

@Qmando
I tried the other way and it worked now. However, I'm not able to use two compare_key in same white list rule as shown below. Any options to achieve this
Below one works with single compare_key : srx-source-zone-name
root@ubuntu:~/SRX_log# more /tmp/elastalert/Network_scan/whitelist-zone.yaml

name: whitelist zone and ip
import: test-srx

type: whitelist

compare_key: srx-source-zone-name

ignore_null: true

whitelist:

  - "!file /opt/whitelist-zone.txt"

Below one with two compare_key: [srx-source-zone-name,  srx-source-address] is not working
name: whitelist zone and ip
import: test-srx

type: whitelist

compare_key: [srx-source-zone-name,  srx-source-address]

ignore_null: true

whitelist:

  - "!file /opt/whitelist-zone.txt"
  - "!file /opt/whitelist-ip.txt"

@sathishdsgithub What is "the other way" which worked? We've tried a bunch of different ways to import a whitelist.yaml which imports a txt file, and it just ignores it.

@Prinzka
If you want to use import, you can only use one. It should look like this https://github.com/Yelp/elastalert/issues/572#issuecomment-315424522
Yes, its a very anaemic feature right now.

@Qmando
We're only using one import file at a time. Yeah it's not great, but if we can chain them we can make it work.
But we are not able to make it so that we can use a whitelist in a frequency type elastalert.
We've tried:
Frequency rule import -> <A yaml file of whitelist type with the whitelist inside the same file. ALso tried with stripping the .yaml.>
Result is that the whitelist is ignored, frequency rule still fires as normal
Frequency rule import -> <A yaml file of whitelist type with the whitelist being "!file /bla/somefile.txt ALso tried with stripping the .yaml.> -> /bla/somefile.txt having the whitelisted computer names in it.
Again, result is that the whitelist is ignored, frequency rule remains functional.
We verified that the !file in the whitelist works by just running a whitelist yaml directly.
However, we want a frequency rule type which we can feed a central file to use for its whitelist.
That sounds like it should be possible, however in practice it is not happening.

You got !file to work? I'm suspect of that. It's not part of pyyaml that I know of.
the whitelist setting only works with type: whitelist, not with frequency. If you want to do that, convert your whitelist into elasticsearch terms.
Here's an example:
type: frequency
filter:
 - not:
      terms:
        ip_adddress:
          - <IP_ADDRESS>
          - <IP_ADDRESS>
          - <IP_ADDRESS>

!file works but only inside a whitelist type rule.
Your comment https://github.com/Yelp/elastalert/issues/572#issuecomment-315424522 suggests that we can import a whitelist file/rule into our frequency rule.
And @sathishdsgithub is implying he got that to work.
We want to be able to use central whitelist files, not have to actually edit every rule that uses it every time there's a change.
I've tried a bunch of different variations with using terms in the filter.
However, using "- not:" inside "filter:" in any form is causing it to error out.

You are using Elasticsearch 5 or 6? You need to do query strings instead unfortunately.
filter:
 query_string:
   query: "NOT (<IP_ADDRESS> OR <IP_ADDRESS> OR <IP_ADDRESS>)"
   default_field: ip_address

Also, so you're completely right about the !file thing. It was added from somebody's pull request last year. I can't even keep track of the features anymore 😢

We're using ES 6.
We're using query strings for the actual query in the rules.
What we would like to do is not have to put our whitelist into the actual query in the rule itself.
What we're looking for is some way to use an external file with a whitelist, in a non-whitelist rule. So that we can have a central file that multiple rules can use and so that it's more easily edited when it comes to temporary whitelists.
The posts in here suggest that we can, but it doesn't seem to function for us.

@Prinzka

You have to create a file without *.yaml in the name of the file (example: simply create whitelist-ip-address) and put your whitelist IP address in below format and save the file in the same rule folder

filter:
   - query:
        query_string:
          query: "NOT srx-source-address: (<IP_ADDRESS> OR <IP_ADDRESS> )"

Use import:   to import the file to your rule . You have to mention the import option in your rule.yaml file
Example:

import:
  - whitelist-ip-address

You can use the same whitelist file in multiple rules

Hello All,
We are working on elastalert rules. Type blacklist not taking wildcard in blacklist files that means when i enter
blacklist:
-"!file /blacklist-file"
In blacklist file we have several list of blacklist text rows like below:
blacklist-file content:
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
Requirement we need to add wildcards inside blacklist textfile which is not working currently.
example requirement below:
blacklist-file content:
<IP_ADDRESS>*
Could you please help us on achieving requirement?
Thank you,
Prabhakar.

If you need to use wildcards/regular expressions, you need to do it in the filter.
type: any
filter:
- query:
     query_string:
       query: "ip_address: /<IP_ADDRESS>.*/"

@Qmando
Thank you so much for quick response. I have just provided example which has sample ips but in real there are many ip_address in blacklist file where as we need to use wildcards inside blacklist-file.  In elastic search these are 10.18. is a dynamic variable we do not know when this will get changed to what. So we need wildcard to search those in elastic search. Since these are many we cannot use type any is there any other solution that you can propose. I am giving blacklist examples below
blacklist-file content:
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
So many IP address available in which last four characters are getting changed so we thought of using regular expression instead of adding multiple entries inside file. This file also contains some of the unique IP address in huge number.
Can we use both blacklist file and query_string in blacklist type rule file?
As per your advise it seems like "!file /blacklist-file" is not required not I have gone through #1753 did not see documentation that explains more about the new change.
Could you please provide if you have documentation link related to this.
Thank you,
Prabhakar.

"Since these are many we cannot use type any"
This doesn't make sense to me. Type any with a filter is the same as blacklist. You can have as many regexs as you want.
filter:
- query:
     query_string:
       query: "ip_address: /10.18\\..*/ OR ip_address: /10.20\\..*/ OR ip_address: /10.55\\.*/"

Regarding the #1753 change, it's actually completely automatic. If you are using a blacklist or whitelist, the filters will be automatically updated. This just makes the elasticsearch queries much smaller and more efficient. If you want multiple rules to inherit the same blacklist/whitelist, you still need to use !file.


RAM overflow error when build app for PineTime

Hi,
I would like to try out a simple demo app using LVGL which was introduced in Mynewt v1.11. Unfortunately I get an error while building project:
$ newt build limon_v1
...
Error: /usr/lib/gcc/arm-none-eabi/10.3.1/../../../arm-none-eabi/bin/ld: /home/(...)/apache-mynewt-newt-bin-linux-1.11.0/limon_v1/bin/targets/limon_v1/app/apps/blinky/blinky.elf section `.bss' will not fit in region `RAM'
/usr/lib/gcc/arm-none-eabi/10.3.1/../../../arm-none-eabi/bin/ld: region RAM overflowed with stack
/usr/lib/gcc/arm-none-eabi/10.3.1/../../../arm-none-eabi/bin/ld: region `RAM' overflowed by 5156 bytes
collect2: error: ld returned 1 exit status

If I remember correctly, I only added @apache-mynewt-core/hw/drivers/display/lvgl into app pkg.yml as a package dependency and set LVGL display resolution and LCD_DC_PIN in the appropriate syscfg.yml file. I did not modify any further files and I use built-in LVGL demos.

@Kisszalimo, while I don't have pinetime watch to verify that it works, here is what you may try:
There is pending PR #3063 that adds options that can tweak LVGL a little bit so it will fit in memory.
You could merge commits from this PR to your mynewt-core branch.
I hope PR will be merged soon.
Next you can change you target syscfg.yml to have this:
syscfg.vals:
    LCD_ITF: spi
    LCD_SPI_DEV_NAME: '"spi0"'
    LCD_SPI_FREQ: 8000
    LCD_SPI_MODE: BUS_SPI_MODE_0
    LCD_CS_PIN: 25
    LCD_RESET_PIN: 26
    LCD_DC_PIN: 18
    LCD_BL_PIN: 13
    LVGL_DISPLAY_HORIZONTAL_RESOLUTION: 240
    LVGL_DISPLAY_VERTICAL_RESOLUTION: 240
    LVGL_ST7789: 1
   # this reduce memory usage of LVGL
    LV_MEM_SIZE: 40000

I did test it on nordic_pca10040 target that has same MCU but no other peripherals except LCD

@kasjer thank you for your response. I added a new repo including the PR you mentioned to my project. On the first try, the problem was not solved:
 Error: /usr/lib/gcc/arm-none-eabi/10.3.1/../../../arm-none-eabi/bin/ld: /home/(...)/apache-mynewt-newt-bin-linux-1.11.0/limon_v1/bin/targets/limon_v1/app/apps/blinky/blinky.elf section `.bss' will not fit in region `RAM'
/usr/lib/gcc/arm-none-eabi/10.3.1/../../../arm-none-eabi/bin/ld: region RAM overflowed with stack
/usr/lib/gcc/arm-none-eabi/10.3.1/../../../arm-none-eabi/bin/ld: region `RAM' overflowed by 248 bytes

However, the amount of overflowed bytes has decreased a lot. So on the second try, I modified my target syscfg.yml file according to your advice, and then the project built successfully.
Before Mynewt supported LVGL, there was an unofficial port to Mynewt. The main difference is the version of LVGL: the unofficial port uses V7, while Mynewt port uses V8. I thought it would be interesting to compare these two ports, so I ran a benchmark demo to test both ports. Unfortunatelly it seems the older port cannot measure the performance properly (or I did something wrong), because it shows very high results (e.g. weighted FPS: 9000), which does not make any sense.
So finally I created two very simple scrolling-label demos using both ports. The functionality and included libraries (except ports) are the same. Strangely the size of generated image files differs significantly:
Image size using Mynewt port: 219KB
Image size using older port: 138KB
I am wondering what causes this big difference and how can we reduce it. Probably the new version of LVGL requires more space... or newt wants to compile "too much thing" into the image?

@Kisszalimo, I checked some more, and it looks like LVGL has now more features that take space.
Here is a way to trim your build.
In your application or target syscfg.yml add line
LVGL_DEFAULT_LV_CONF: 0

This will prevent usage of mynewt provided lv_conf.h file which can customize some of LVGL settings from mynewt-like syscfg.yml files but it does not cover most of them, only some basic configuration that is ok to run LVGL demos.
Copy hw/drivers/display/lvgl/default/lv_conf.h file to your application include directory, and disable all LVGL features that are not needed for your scrolling-label demo (maybe calendar, arc and such)
If you want to execute LVGL demos that are covered by apps/lvgl/demo with your customized lv_conf.h you can add include directory to folder where your target is located (i.e. targets/pinetime-lvgl_demo/pkg.yml) add line
pkg.cflags:
    - -Iinclude

That will add include to all compiled files and then you can have your lv_conf.h that can be fully customized with all the features that LVGL exposes.

@kasjer i was able to reduce my build by applying your advice. Now it uses ~180KB space which is a notable difference (of course it depends on the needs of your application). However if flash memory is a bottleneck, maybe you should consider using an older verson of LVGL.
I am fairly new to mynewt and sometimes the project structure and its working mechanism is a bit mysterious for me. For example, if I do not add
LVGL_DEFAULT_LV_CONF: 0

to my app syscfg.yml (leave it default), it seems mynewt use my custom lv_conf.h anyway, instead of ignoring it and use the default LVGL config file as expected. The only way to force mynewt to use default config file is to delete my custom config file from the project.

For now there is no effort to have older version supported.
If I have time I will see if we can override LVGL version in project.yml (I'm not sure if this will require some work on newt tool though).
You are right about lv_conf.h if you have one added to application or target it will automatically override any file with the same name in the project. For now this is the case because we include it by #inlude <lv_conf.h>. LVGL have more options to specify config file that I may try out once all things are settled.
I'm not expert on LVGL by any means so if you want to contribute to mynewt project in this area your contribution will be appreciated.


Bug Restricting Capacities and Volumes

In additional_functionality the following constraints are not exactly working as intended.

FT production limit
capacity limits for electrolysis in Germany
since the limit is set on the capacity of bus0 which in the case of FT production is hydrogen and in case of electrolysis is electricity. Dividing the capacity by the efficiency should solve the bug.

closed by #278 and direct follow-up commits to main


v3 proposal

It extends #24 and add php 7.1 support
ping @weierophinney @Maks3w @bakura10

@snapshotpl
I see some problems, because you've mixed different problems in one commit:

this breaks the version control system
no way for a cherry-pick
reading the commit history is a mess
a review is very difficult

Each individual change should be one commit. Please create separate commits for each problem. Thank you in advance!

@snapshotpl Totally missed this when I submitted #181 earlier.
@froschdesign For large refactors, we simply cannot do one change at a time. In such cases, I would recommend posting a WIP with some of the ideas codified, and then creating an RFC in the contributors section of the forums. I'll be posting one related to #181 tomorrow, as I've done some significant work trying to address forwards and backwards compatibility at this point.

@weierophinney

For large refactors, we simply cannot do one change at a time.

You can, because a problem can change more than one line or one file!

In this PR, there's a single commit that does two different things:

Updates travis.yml and composer.json to PHP 7.1
Adds Result.php, Message.php and the changes ValidatorInterface.php.

It seems to me that these should be two separate PRs. I agree with Matthew that more explanation on the reasons behind a significant change would be useful.

@akrabat

I suppose it all depends on what a "change" is.

Replace the word "change" with "problem".

In this PR, there's a single commit that does two different things

Right!

I agree with Matthew that more explanation on the reasons behind a significant change would be useful.

I agree also.

I wanted to point out the problem with the "all-in-one-commits". No more.
We should add a new paragraph to the contributors guide to respect the principles of a version control system.

This repository has been closed and moved to laminas/laminas-validator; a new issue has been opened at https://github.com/laminas/laminas-validator/issues/17.

This repository has been moved to laminas/laminas-validator. If you feel that this patch is still relevant, please re-open against that repository, and reference this issue. To re-open, we suggest the following workflow:

Squash all commits in your branch (git rebase -i origin/{branch})
Make a note of all changed files (`git diff --name-only origin/{branch}...HEAD
Run the laminas/laminas-migration tool on the code.
Clone laminas/laminas-validator to another directory.
Copy the files from the second bullet point to the clone of laminas/laminas-validator.
In your clone of laminas/laminas-validator, commit the files, push to your fork, and open the new PR.
We will be providing tooling via laminas/laminas-migration soon to help  automate the process.


Add basic grid functionnalities

Add grid to be able to handle costs
Switch to Python 3.9
Better typing

Codecov Report

Merging #14 (0d959b8) into main (402dfcd) will increase coverage by 0.25%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #14      +/-   ##
==========================================
+ Coverage   98.82%   99.08%   +0.25%
==========================================
  Files           3        4       +1
  Lines          85      109      +24
  Branches        7       12       +5
==========================================
+ Hits           84      108      +24
  Partials        1        1

Impacted Files
Coverage Δ

src/easygrid/microgrid.py
98.52% <100.00%> (+0.22%)
:arrow_up:

src/easygrid/types.py
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Privacy manifest

Should we have a privacy manifest?
I've seen Apple requires one if you use AppAuth
I can take an early stab at it.

Well that was easy ;)

We all like easy ;)


Blocking rules don't work alongside with specific $replace exceptions rules

Steps to reproduce

Add these rules to user filter:
||google.com^$replace=/try/\/*test*\/try/
@@||google.com/xjs/$replace=/try/\/*test*\/try/
||google.com/xjs/$important
Go to https://www.google.com/

Expected behavior

Requests from google.com/xjs/ should be blocked.
Actual behavior

Requests from google.com/xjs/ are not blocked.
Screenshot:

It works fine if I use @@||google.com/xjs/$replace (instead of @@||google.com/xjs/$replace=/try/\/*test*\/try/)
Your environment

AdGuard for Windows 7.2.2910 (CL 1.4.157)

Needs to be resolved with #1106

fixed along with #1106


Add ekobi A65 14W + Sylstar 24W

Do you have some more information about this exact light? As A65 14W seems quite generic.
Also please check the supported_color_modes of the light.
I see you included both brightness and color_temp, but this combination should not be possible in HA, also according to the specs here: https://developers.home-assistant.io/docs/core/entity/light/#color-modes.
You should probably only provide either brightness or color_temp. The first one when only brightness can be controlled in HA, second when it's possible to change the color temperature in HA.

Thanks for the quick feedback. The light supports RGB and color temperature
adjustment. Unfortunately I could not get readings for HS mode. I'll get a
Tape P110 on Monday to see if I can get accurate values.
As for the model, it's this one: ekobi SMART LED E27 14W 1400Lm RGB CCT
WIFI TUYA, Alexa, Android, iOS, google home, Sprachsteuerung, Smartphone
App Steuerung, Mehrfarbige Birne, Dimmbar, Frequenz 2,4GHz 2700-6500K, 3-er
Pack https://amzn.eu/d/8r246V6
Bram Gerritsen @.***> schrieb am So., 3. Sept. 2023,
08:41:

Do you have some more information about this exact light? As A65 14W seems
quite generic.
Also please check the supported_color_modes of the light.
I see you included both brightness and color_temp, but this combination
should not be possible in HA, also according to the specs here:
https://developers.home-assistant.io/docs/core/entity/light/#color-modes.
You should probably only provide either brightness or color_temp. The
first one when only brightness can be controlled in HA, second when it's
possible to change the color temperature in HA.
—
Reply to this email directly, view it on GitHub
https://github.com/bramstroker/homeassistant-powercalc/pull/1852#issuecomment-1704027100,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHXIGOUVXY4BZPW74MPPELXYQRBVANCNFSM6AAAAAA4JBWBPY
.
You are receiving this because you authored the thread.Message ID:
@.***>

In that case you'll need to provide both hs and color_temp.
Let's await till you have the Tapo, hopefully that one will be able to get some measurements.
You can set SMART LED E27 14W 1400Lm RGB CCT in name property of model.json as that will be shown in supported model listing.

@bramstroker I've updated this to use Tapo P110 and used color_temp and hs modes. I've also added a new ceiling lamp which is this model

@forabi Thanks.
In the future, could you please create a single PR per light? You'll need to make a new branch in git for that. plenty of tutorials to found on internet for that.
 These are only needed for auto discovery. I see you put a very long name in it, that can't be correct.
Please have a look into device page of the light, and the information on the top left.

Could you share this information for both of the lights please?

@bramstroker Updated the aliases

Thanks. Still left with some questions though.
As I cannot find anything about uebyc181yivkcndi and bcyytn6qryldlgax on Google, seems not an actual model or something we can use to autodiscover the same model on other users sytems.
Smart Ceiling Lamp seem also way to generic and not an actual model identifier.
This whole Tuya ecosystem is such a big mess unfortunately.
Could you share the device information as seen in HA as shown on my screenshot in previous comment?
Also which integration are you using to get these lights into Home Assistant?

@bramstroker This is a Tuya model ID, the auto discovery works with my branch just fine.

@bramstroker This is a Tuya model ID, the auto discovery works with my branch just fine.

For the Sylstar light according to following amazon link the model is SL-CL-24, which seems an actual model number handled by the brand. So I suggest to change the directory name from Smart Ceiling Lamp to SL-CL-24 so we have correct model in supported model listing: https://github.com/bramstroker/homeassistant-powercalc/blob/master/docs/supported_models.md
https://www.amazon.co.uk/SYLSTAR-Ceiling-Dimmable-Assistant-Bathroom/dp/B09HH1GTQK

@bramstroker This is a Tuya model ID, the auto discovery works with my branch just fine.

I don't know Tuya that well as I don't have any devices myself, but maybe this hash is unique per Tuya device in each unique Tuya installation?
It's enough to only add uebyc181yivkcndi and bcyytn6qryldlgax as alias because Powercalc will check value between parentheses as fallback.
But when this hash is unique per installation it will work for your HA installation, but not for others, so makes no sense to add the alias in that case. But not sure how to know this for sure.

@bramstroker I believe you are right, this is probably the device virtual ID which is unique per each device "boostrap" process. I'll see if I can find some more stable model ID

@bramstroker Found this in the diagnostics file for Tuya integration:
    "name": "Bedroom Lamp",
    "model": "SYS-C24W-CW",
    "category": "dj",
    "product_id": "bcyytn6qryldlgax",
    "product_name": "Smart Ceiling Lamp",

The other lamp:
    "name": "Bulb 9",
    "model": "",
    "category": "dj",
    "product_id": "uebyc181yivkcndi",
    "product_name": "A65  14W    \u4e94\u8def   BK-N   \u516c\u7248 ",

@bramstroker I believe you are right, this is probably the device virtual ID which is unique per each device "boostrap" process. I'll see if I can find some more stable model ID

Ah, that clarifies why I couldn't find any reference on Google.
But when this is the only information which is exposed to HA we cannot use it for auto discovery.
So I don't think any stable model id is exposed to HA somehow which can be utilized.
That being said we can still adopt the lights into the DB, but the user must set it up themself using this procedure: https://homeassistant-powercalc.readthedocs.io/en/latest/sensor-types/virtual-power-library.html.

Not really much more to work with unfortunately looking at those diagnostics.
That very unfortunate but we have to work with what is available.
We cannot provide reliable autodiscovery for these lights. I might add a boolean to model.json in the future to indicate that, so we can also list in the supported model list that these lights cannot be autodiscovered.
To get the PR wrapped up following needs to be resolved:

You may choose SYS-C24W-CW or SL-CL-24 for the directory name of the Sylstar
aliases must be removed from the model.json as these are not applicable to other users of powercalc
Other suggestion P110 -> Tapo P110 applied.
Name of the ekobi light set to ekobi SMART LED E27 14W 1400Lm RGB CCT

I was still search for an actual model id for the ekobi, but was not able to find any.
Maybe on the package? But I doubt you still have that.

Ekobi

@bramstroker Addressed all feedback, also renamed ekobi's folder to the model name on the package

Thanks for your efforts. Looks good now.


modified:   src/providers/linkedin.ts

The id_token was missed from validateAuthorizationCode()
The actual fetch response is like:
{
  access_token: 'xxxxxxxxx',
  expires_in: 5183999,
  scope: 'email,openid,profile',
  token_type: 'Bearer',
  id_token: 'xxxxxxxxx'
}

The refresh token is available only if your application is authorized for programmatic refresh tokens according to:
https://learn.microsoft.com/en-us/linkedin/shared/authentication/programmatic-refresh-tokens

Can you create a changelog? Should be a minor
https://github.com/pilcrowOnPaper/arctic/blob/main/CONTRIBUTING.md#creating-changesets

Thanks!