Adapting downward metrics to feature lifecycle

For a VM to access the downward metrics, it requires defining the downwardMetrics device:
(i.e.,  spec.domain.devices.downwardMetrics: {}):
apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
[...]
spec:
  domain:
    devices:
      downwardMetrics: {}
[...]

see: https://kubevirt.io/user-guide/storage/disks_and_volumes/#virtio-serial-port
Currently, the downward metrics feature is behind the feature gate DownwardMetrics.
In order to enable the downward metrics feature, the user needs to add that feature gate:
apiVersion: kubevirt.io/v1
kind: KubeVirt
[...]
spec:
  certificateRotateStrategy: {}
  configuration:
    developerConfiguration:
      featureGates:
      - DownwardMetrics
[...]

Is your feature request related to a problem? Please describe:
Since the downward metrics is already GA, the feature gate must be removed according to the KubeVirt Feature Lifecycle document. However, we need to enabled/disabled it at any time as discussed on https://issues.redhat.com/browse/CNV-38687
Describe the solution you'd like:
To enable/disable the downward metrics at cluster level, we could add the spec.configuration.downwardMetrics: {}  (disabled by default).
Therefore, there are two requisites that needs to be satisfied in order to enable the downward metrics: to add spec.configuration.downwardMetrics: {} to KubeVirt CR and the spec.domain.devices.downwardMetrics: {} device to VM object.
This opens some questions about the potential scenario combinations:

What should happen when the VM defines spec.domain.devices.downwardMetrics: {}, but spec.configuration.downwardMetrics: {} it is not present in the KubeVirt CR?

IMO the VM manifest should be accepted (in contrast with what the current vmi admitted does if the FG is not enabled), and when the VM is requested to be started, mark it as FAILED showing the reason.

Or/And should we raise an alert?

If the cluster admin tries to remove spec.configuration.downwardMetrics: {}, while are VMs running with spec.domain.devices.downwardMetrics: {}. Should we reject this update?

Additional context:
Effort tracked on https://issues.redhat.com/browse/CNV-34216
Looking for your feedback.
Thanks in advance.

Unfortunately I cannot access CNV-38687. Can you please explain why we need an option for this feature to be guarded by the cluster admin?
/cc  @EdDev FYI
Your work is bringing some fruit

Can you please explain why we need an option for this feature to be guarded by the cluster admin?

+1
I think we should strive to have as little as possible knobs on the API.
If someone does not want this option on the VM, it should not be define there in the first place.
But this is just a general direction, each case should be negotiated with convincing reasoning.

If the cluster admin tries to remove spec.configuration.downwardMetrics: {}, while are VMs running with spec.domain.devices.downwardMetrics: {}. Should we reject this update?

I would not recommend it. It will require scanning all VMIs and the result will not assure there is no VMI running with this option. The async nature of the system provides no reliable way to do this in a simple way.

The downward metrics feature exposes some metrics about the host node where the VM is running to the guest. This information may be considered sensitive information. You can find an example of what kind of information is exposed here:
<metrics>
  <metric type="string" context="host">
    <name>HostName</name>
    <value>node01</value>
  </metric>
  <metric type="string" context="host">
    <name>HostSystemInfo</name>
    <value>linux</value>
  </metric>
  <metric type="string" context="host">
    <name>VirtualizationVendor</name>
    <value>[kubevirt.io](http://kubevirt.io/)</value>
  </metric>
  <metric type="string" context="host">
    <name>VirtProductInfo</name>
    <value>QEMU 7.2.0</value>
  </metric>
  <metric type="real64" context="vm" unit="s">
    <name>TotalCPUTime</name>
    <value>30.260000</value>
  </metric>
  <metric type="uint64" context="vm" unit="KiB">
    <name>PhysicalMemoryAllocatedToVirtualSystem</name>
    <value>1000000</value>
  </metric>
  <metric type="uint64" context="vm" unit="KiB">
    <name>ResourceMemoryLimit</name>
    <value>1000000</value>
  </metric>
  <metric type="int64" context="host">
    <name>NumberOfPhysicalCPUs</name>
    <value>1</value>
  </metric>
  <metric type="real64" context="host" unit="s">
    <name>TotalCPUTime</name>
    <value>258.480000</value>
  </metric>
  <metric type="uint64" context="host" unit="KiB">
    <name>FreePhysicalMemory</name>
    <value>4918944</value>
  </metric>
  <metric type="uint64" context="host" unit="KiB">
    <name>FreeVirtualMemory</name>
    <value>4918944</value>
  </metric>
  <metric type="uint64" context="host" unit="KiB">
    <name>MemoryAllocatedToVirtualServers</name>
    <value>3791488</value>
  </metric>
  <metric type="uint64" context="host" unit="KiB">
    <name>UsedVirtualMemory</name>
    <value>3791488</value>
  </metric>
  <metric type="uint64" context="host" unit="KiB">
    <name>PagedInMemory</name>
    <value>0</value>
  </metric>
  <metric type="uint64" context="host" unit="KiB">
    <name>PagedOutMemory</name>
    <value>0</value>
  </metric>
  <metric type="int64" context="host" unit="s">
    <name>Time</name>
    <value>1685031704</value>
  </metric>
  <metric type="uint64" context="vm">
    <name>ResourceProcessorLimit</name>
    <value>2</value>
  </metric>
</metrics>

This discussion was triggered because in HCO the feature gate DownwardMetrics was enabled by default. Therefore, any VM could enable the metrics and inspect information that, in some cases, the admin would like to hide, this was reported as a security issue. In the past, we patch this by not enabling the feature gate DownwardMetrics by default and opening a mechanism to enable the feature gate (making it opt-in). Further details can be found at: CVE-2024-31419, https://github.com/kubevirt/hyperconverged-cluster-operator/pull/2823 and https://github.com/kubevirt/hyperconverged-cluster-operator/pull/2844
Therefore, if we want to get rid of the feature gate DownwardMetrics to adapt this feature to the new feature lifecycle, we need a way to enable/disable the downward metrics feature. Otherwise, we will fall in the same security issue.
@geetikakay @sarahbx Your input would be much appreciated. Thanks!

@geetikakay @sarahbx Your input would be much appreciated. Thanks!

I think you summed it up well!

To enable/disable the downward metrics at cluster level, we could add the spec.configuration.downwardMetrics: {} (disabled by default).
Therefore, there are two requisites that needs to be satisfied in order to enable the downward metrics: to add spec.configuration.downwardMetrics: {} to KubeVirt CR and the spec.domain.devices.downwardMetrics: {} device to VM object.
This opens some questions about the potential scenario combinations:
1. What should happen when the VM defines `spec.domain.devices.downwardMetrics: {}`, but `spec.configuration.downwardMetrics: {}` it is not present in the KubeVirt CR?

* IMO the VM manifest should be accepted (in contrast with what the current vmi admitted does if the FG is not enabled), and when the VM is requested to be started, mark it as FAILED showing the reason.

* Or/And should we raise an alert?

I would support accepting the VM, marking it as failed due to an invalid configuration, and raising an alert. However, the failing to start could be a problem for running VMs, continued below. The failed state alone might be enough for the user, but the alert would be useful for the admin.

2. If the cluster admin tries to remove `spec.configuration.downwardMetrics: {}`, while are VMs running with `spec.domain.devices.downwardMetrics: {}`. Should we reject this update?

I would not reject the update. The configuration could be accepted, the downwardMetrics server disabled, and allow existing VMs to keep running until restart. The socket will fail, requiring to stop, edit, and restart to disable the device.
The process within the VM utilizing the socket would have to handle the disabled case properly.
If however, it makes sense to provide the configuration as a simple on/off switch, allowing the VM to run in either state, I would suggest adding aditional fields for limiting usage within the cluster.
eg:
.spec.configuration.downwardMetrics: {} == disabled
.spec.configuration.downwardMetrics:
  namespaces:
    - user1ns:
        serviceAccount: user1-sa

IMO the VM manifest should be accepted (in contrast with what the current vmi admitted does if the FG is not enabled), and when the VM is requested to be started, mark it as FAILED showing the reason.

Why?

If the cluster admin tries to remove spec.configuration.downwardMetrics: {}, while are VMs running with spec.domain.devices.downwardMetrics: {}. Should we reject this update?

Why?
Imho we should treat only admission of VMs.

IMO the VM manifest should be accepted (in contrast with what the current vmi admitted does if the FG is not enabled), and when the VM is requested to be started, mark it as FAILED showing the reason.

Why?

I might be wrong, but my reasoning is because in this way the checking will be closer to the responsible code.

If the cluster admin tries to remove spec.configuration.downwardMetrics: {}, while are VMs running with spec.domain.devices.downwardMetrics: {}. Should we reject this update?

Why?

Because the cluster will be in an inconsistent state, i.e., some VM will be running with DM enabled while the general configuration will state that DM should be not allowed.

Imho we should treat only admission of VMs.

Won't this force us to move the checking to the webhooks, causing the same issues explained in Eventual Consistency from First Principles
?

@jcanocan I suggest the following

This opens some questions about the potential scenario combinations:
What should happen when the VM defines spec.domain.devices.downwardMetrics: {}, but spec.configuration.downwardMetrics: {} it is not present in the KubeVirt CR?
IMO the VM manifest should be accepted (in contrast with what the current vmi admitted does if the FG is not enabled), and when the VM is requested to be started, mark it as FAILED showing the reason.
Or/And should we raise an alert?

One possible solution is to have the VM raise an alert about the invalid configuration and simply ignore the related spec fields. This means that when the VM starts, it would disregard the spec.domain.devices.downwardMetrics: {} field, effectively removing it from the configuration. This approach would allow the VM to continue running, while also notifying the administrator about the invalid configuration.

If the cluster admin tries to remove spec.configuration.downwardMetrics: {}, while are VMs running with spec.domain.devices.downwardMetrics: {}. Should we reject this update?

As a user, I would be dissatisfied if this type of update were rejected. To address this, we could either provide a warning about the potential impact or document it.
The VM can continue to function normally, but whenever it attempts to extract metrics about the host node, alerts should be raised to notify the admin. This way, even if the FG is disabled, the admin will be aware that a VM is still running and extracting metrics, allowing them to make an informed decision.
Additionally, a VM with a downward metrics specification should be able to stop regardless of whether the FG is set to true or false. Tracker

@geetikakay Just one note, the idea is to completely remove the Feature Gate and enable a new configuration spec in the KubeVirt CR

If the cluster admin tries to remove spec.configuration.downwardMetrics: {}, while are VMs running with spec.domain.devices.downwardMetrics: {}. Should we reject this update?

I would not recommend it. It will require scanning all VMIs and the result will not assure there is no VMI running with this option. The async nature of the system provides no reliable way to do this in a simple way.

+1
If we want to go the eventually consistent route, this is one of the bigger issues to solve. Preventing a VM from starting until the feature was enabled is easy, but preventing disabling the feature while a VM still makes use of it is hard.
I would agree that we should try to move from rejecting the VM to not starting it until the feature is enabled. It would allow us to come one step closer to an eventual consistency model in this project. However, I do not agree with @geetikakay's proposal, a VM should not start if its spec cannot be satisfied.

+1

The big question to me is, what should happen if the feature is disabled while still in use? Basically, all of this applies to other features / FGs in KubeVirt too. How can we move from features being guarded by webhooks to an eventual consistent model? Before making any changes in this special case we should agree on a common way forward for the whole project first.

This is a good point, IMHO.
Thanks for your input @0xFelix!

That's exactly what I was talking about. We're not able to satisfy the spec here so I don't think the VM should start.

That's exactly what I was talking about. We're not able to satisfy the spec here so I don't think the VM should start.

+1

The big question to me is, what should happen if the feature is disabled while still in use?
Basically, all of this applies to other features / FGs in KubeVirt too. How can we move from features being guarded by webhooks to an eventual consistent model? Before making any changes in this special case we should agree on a common way forward for the whole project first.

k8s is often pretty naive on this affecting only scheduling but not the existing workload:
for instance if you have a daemonset with existing pods, nothing is preventing you from reconfiguring it and the pods scheduled in the past according to the previous configuration are not reclaimed but simply eventually counted as  numberMisscheduled until they are going to die for external reasons without getting rescheduled due to the updated configuration.

@tiraboschi That sounds acceptable to me if we follow Kubernetes.
But again, before looking at this particular instance we should find an agreement in the grand scheme of things first. How should this work for all FGs / configurables in the project? I'll look into writing a design proposal for that. Once we know how to proceed for the whole project we can begin implementing it. Therefore I'd vote to pause work on this for a bit.

Let's summarize what has been discussed so far regarding open questions:

What should happen when the VM defines spec.domain.devices.downwardMetrics: {}, but spec.configuration.downwardMetrics: {} it is not present in the KubeVirt CR?

The VM should not start at all, marking it as FAILED and showing a clear message that the configuration it is not valid/allowed.

Or/And should we raise an alert?

Yes. It should be raised to notify the sysadmin.

If the cluster admin tries to remove spec.configuration.downwardMetrics: {}, while are VMs running with spec.domain.devices.downwardMetrics: {}. Should we reject this update?

The better argued option is to let the VM run and serve the Downward Metrics until it is manually rebooted, following Kubernetes.
However, these aspects need to be discussed in a specific design proposal for configurable features.

I am not sure what we gain by not rejecting the VM in the first place. I am afraid we are over-complicating here.

/close


fix:#898 Update index.html

原cdn.bootcss.com域名失效，需要切换到新域名cdn.bootcdn.net
官网申明：https://www.bootcdn.cn
The branch of 0.2.0 is master-0.2.0. If your PR is to solve the problem of 0.2.0, please submit the PR to it.
Please do not create a Pull Request without creating an issue first.
What is the purpose of the change
XXXXX
Brief changelog
XX
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist.
[ ] Run `mvn clean compile --batch-mode -DskipTests=false -Dcheckstyle.skip=false -Drat.skip=false -Dmaven.javadoc.skip=true to make sure basic checks pass.

Codecov Report

Merging #899 (78500e1) into develop (e44d831) will decrease coverage by 0.04%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             develop     #899      +/-   ##
=============================================
- Coverage      21.61%   21.57%   -0.05%
+ Complexity       427      426       -1
=============================================
  Files            149      149
  Lines           4686     4686
  Branches         712      712
=============================================
- Hits            1013     1011       -2
- Misses          3590     3593       +3
+ Partials          83       82       -1

Flag
Coverage Δ

unit
21.57% <ø> (-0.05%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...apache/dubbo/admin/service/RegistryServerSync.java
87.50% <0.00%> (-2.78%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e44d831...78500e1. Read the comment docs.


Multishift qz

Changes in this PR:

implement inverse householder reflector of order 3.
Use inverse householder in unblocked QZ. This resulted in a surprizing amount of speedup. Almost 30%
implement schur swap (this has a significant change w.r.t. lapack, because i use 2x2 svd and givens instead of QR), should be significantly more accurate, but i need to do some benchmarks to see if it is also fast.
implement multishift sweep
implement schur move
implement AED
implement full multishift QZ

A big design change from LAPACK is that this implementation is based on the multishift QR implementation. This should make maintenance easier. This also means that unlike the LAPACK implementation, I do use subdiagonal entries as scratch space in AED and multishift sweep. In QR, this accounts for almost all of the workspace, except for the delayed updates. In QZ, there are no delayed updates so no workspace is required at all.
Some timings:

n
TLAPACK
LAPACK

250
0.053
0.064

500
0.180
0.172

1000
0.747
0.798

2000
2.32
2.47

4000
10.8
15.0

So we are a bit faster than LAPACK, probably mostly because of the inverse Householder reflectors.

The merge was to resolve a conflict with a CMakeLists.txt


chore: rename ?worker-runner to ?worker-env

#121  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @hi-ogawa and the rest of your teammates on  Graphite


Update motor controller configuration

CAN SparkMax Controllers
Implementation Plan
Either add or update existing motor objects to create a differential drivetrain with 2 left motors and 2 right motors, where one motor is in follow mode. The motor controllers are CAN based SparkMax and the motors are Rev Neo.
Affected Subsystem(s):

Drivetrain

Details:
REV SparkMax API: https://codedocs.revrobotics.com/java/com/revrobotics/cansparkbase
See DifferentialDrive API docs for information on how to combine the motors into a drive with available tankDrive and arcadeDrive methods.

Here's an example of using SparkMax leader/follower:
https://www.chiefdelphi.com/t/new-to-neo-drive-train-what-is-best-with-java-motor-group-or-leader-follower/427142/4

@SarahPick you can update/close this issue now.


chp svelte: Error: EISDIR: illegal operation on a directory, read

I have the following chompfile.toml in an empty directory (or, with src and dist folders--same result):
version = 0.1

[[task]]
  name = "svelte"
  template = "chomp:svelte"
  [task.args]
    indir = "src"
    outdir = "dist"

When I type chp svelte I get the following error (and I don't understand what's going on):
- package.json [- 303.066µs]
- node_modules/svelte [- 702.144µs]
- node_modules/mkdirp [- 700.336µs]
○ :svelte
node:internal/process/esm_loader:94
    internalBinding('errors').triggerUncaughtException(
                              ^

[Error: EISDIR: illegal operation on a directory, read] {
  errno: -21,
  code: 'EISDIR',
  syscall: 'read'
}

See https://github.com/guybedford/chomp/issues/17#issuecomment-1007134652, the process.env.TARGET is undefined, so the JS is trying to read the base directory as the svelte file or something like that I think.


fix test_layerHelpers unittest error

Changes Unknown when pulling 62c5389bad0191c8574d7be58fc1564567b913ef on QiJune:feature/sppnet into ** on baidu:develop**.


[SPARK-30993][FOLLOWUP][SQL] Refactor LocalDateTimeUDT as YearUDT in UserDefinedTypeSuite

What changes were proposed in this pull request?

Refactor LocalDateTimeUDT as YearUDT in UserDefinedTypeSuite
Why are the changes needed?

As we are going to support java.time.LocalDateTime as an external type of TimestampWithoutTZ type https://github.com/apache/spark/pull/32814, registering java.time.LocalDateTime as UDT will cause test failures: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139469/testReport/
This PR is to unblock https://github.com/apache/spark/pull/32814.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

Unit test.

Test build #139500 has started for PR 32824 at commit 7e2c102.

Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44023/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44023/

Merging to master

Hi, @gengliangwang .
SPARK-30993 was released long time ago by 2.4.6 and 3.0.0.
If you don't mind, could you use a new JIRA instead of the follow-up next time?
I'm not aware of the community rule for this. Usually, I use a follow-up when it will have the same release version which means the original patch is not released yet.

@dongjoon-hyun Yes, I think creating a new Jira makes more sense. Thanks for the suggestion!


How to access content header YAML settings in template via pages function

Hello,
First time trying to work with picocms.
In a nutshell, I've pasted the code snippet from the guide on how to build blogging functionality (here: http://picocms.org/docs/#blogging ) into a template. I can see the for loop iterating through all pages in the specified folder and that within the loop we reference page.title, page.date_formatted and page.description. If I understand correctly, these are entries in the YAML header of the content page being evaluated on the given loop.
However, if I wish to add my own header elements - say "ModuleRoot: true" to the index.md - how can I work with that value within the loop? Note - I am using index.md instead of blog.md.
Above is the general question. Below are specific issues I'm encountering.
Currently the loop is listing the index.md page in addition to all the other content. I don't want this. I can't use Hidden: true in the header for index.md because that ends up removing it from my site navigation. I'd like to add a variable/parameter to the header, then check, within that loop, whether that parameter is set (ie. ModuleRoot: true) and if so, avoid rendering that page in the list. I can see the current code checks for not page.hidden but adding not page.ModuleRoot does not work.
As a separate issue, The way I want to set up this pseudo-blogging feature is that the pages within my "blog" folder are not full articles. Instead, the page content is, in fact, the excerpt to be displayed in a list of articles. I want the Title attribute (text) from the header to link to a page elsewhere in the site. That is, the index.md will list all the pages in this folder, but clicking on an entry takes you elsewhere in the site.
I can think of two ways to do this:

I could set up additional parameters in the header for each content item in that folder - things like the URL to which you should be taken when clicking on this item. Then I could modify the template that renders the list of items from this folder so that it generates a link using that URL. I have two problems with this: first, I cannot seem to access custom header parameters; second, I actually want rich content to be shown - so for example, an image which is hyperlinked, has ALT text, has a TITLE attribute, then a paragraph of text - in addition to the article title which is also linked, then a separate paragraph showing the date. Putting all these attributes into separate parameters in the page header would work and would produce consistent formatting for each entry, but it is very cumbersome to maintain. So rather, I prefer the next option:

I could display the content portion of each page in that folder, within the list being generated by this for loop. To do that, I need to reference "page.content", as it were. That does not work. If I reference just "content", I get the content from index.md - which is the page running through that for loop. Is it possible to iterate through every content page and access the content from that page?

Lastly a suggestion - so far, the only docs I could find about the pages function (here: http://picocms.org/in-depth/features/pages-function/ ) - to try and understand what might be output, for example, is an article that reads more like a conversation between one developer coaching another. (The whole "family" and "flower shop" metaphor thing just doesn't work for me. And for example - how can I know if this CMS has other functions accessible to me?)
I think it would be great if there were documentation that was more formalised. Compare, for example, any documentation on any SQL function or statement with any reasonable database vendor - it lists all syntax, optional parameters and explains each parameter, return value, etc and even describes which versions of the product support the function.
On that last note, I don't recall now where I sourced the following line of code:
for page in pages|sort_by("time")|reverse
However I cannot find any documentation about the syntax of using a pipe character with the pages function, nor the sort_by function/parameter, nor the reverse parameter, etc. This is where a formalised reference would help.
Oh, and the blogging documentation uses page.date_formatted for display but instructs to set a parameter named date in the header - why the mismatch? Is date_formatted a parameter name or a special keyword that formats something named "date"? What is the correct format to enter the date against that parameter in the header? These kinds of questions would probably be answered by a formal reference, too, that talks about the YAML content header - and the way in which the pages function uses it - in more detail.
One more PS: I think even the for loop construct, and the conditional if logic, and the comparative operators like 'not' (what about greater than? null?) would also benefit from a formal doc.
Thanks.

Ok - I solved one problem - excluding the index.md file from the list generated by the for loop, using the Twig reference documentation and the following:
and not (page.id ends with "index")
Now for accessing custom values from the page header and/or displaying the page content for each 'page' in the loop...

And problem two - after searching through PHP code I found a useful comment for the function readPages:
    /**
     * Reads the data of all pages known to Pico
     *
     * The page data will be an array containing the following values:
     *
     * | Array key      | Type     | Description                              |
     * | -------------- | -------  | ---------------------------------------- |
     * | id             | string   | relative path to the content file        |
     * | url            | string   | URL to the page                          |
     * | title          | string   | title of the page (YAML header)          |
     * | description    | string   | description of the page (YAML header)    |
     * | author         | string   | author of the page (YAML header)         |
     * | time           | int      | timestamp derived from the Date header   |
     * | date           | string   | date of the page (YAML header)           |
     * | date_formatted | string   | formatted date of the page               |
     * | hidden         | bool     | this page shouldn't be visible           |
     * | raw_content    | string   | raw, not yet parsed contents of the page |
     * | meta           | string[] | parsed meta data of the page             |
     * | previous_page  | &array[] | reference to the previous page           |
     * | next_page      | &array[] | reference to the next page               |
     * | tree_node      | &array[] | reference to the page's tree node        |
     *
     * Please note that the `previous_page` and `next_page` keys don't
     * exist until the `onCurrentPageDiscovered` event is triggered
     * ({@see Pico::discoverPageSiblings()}). The `tree_node` key doesn't
     * exit until the `onPageTreeBuilt` event is triggered
     * ({@see Pico::buildPageTree()}).
     *
     * @see Pico::sortPages()
     * @see Pico::discoverPageSiblings()
     * @see Pico::getPages()
     */

This describes raw_content - which renders the raw markup. Elsewhere I saw that you can use the pipe character to specify when content is raw, thus: {{ page.raw_content|raw }} ... but quickly learned this relates to raw HTML, not markup.
However that gives me the workaround - just code these "blog" pages (really just excerpts to be listed) using straight-up HTML. It works.
And I guess the final answer is that to return custom YML variables in the pages object, I need to customise the above function. So the new question becomes:
How best would I customise that function? Should I be hacking around in Pico.php? Or is there a general approach to extending these functions in a manner that doesn't touch the core file? What would that approach look like?

Ok, pushing a little further, the final answer comes through. The function returns a string array named meta. So here is the solution:
{% if not page.meta.ModuleRoot %}

Not the module root

{% endif %}

(Incidentally, if I set the header value to the text "true", this return value is "1").
This answers all my questions, including to say that the pages object does not return the formatted content, but it can return the raw content. Is there a function I can call in my template that will parse the raw content and render formatted content?

Accessing meta data:
{{ page.meta.My_Custom_YAML_data }}

Parsing contents of another page:
{{ "sub/page"|content }}

About Twig templating see https://twig.symfony.com/doc/1.x/
About creating your own custom theme and Pico's additional Twig features see http://picocms.org/docs/#themes
Not sure whether you've got more open questions, so please don't hesitate to go ahead :smiley:


Provide StructureOfUuids.md (fixes #136)

The applicationName and releaseNumber became relevant with ApplicationLayerTopology application coming into existence.
The specification has often been made without applicationName and releaseNumber and it had been added later.
For future applications the information is required.
And this should also be expressed in the guidelines.

UUIDs
It is still not clear to me, if the already existing UUIDs inside the serviceList (i.e. currently without the appName & releaseNumber) are still valid or if we should not have a new template (or will those be kept like they are all the time and the UUID format with prefix is only relevant for the new services...). So if we are still keeping the template without the prefix for now, but already add uuids following the new format for our own services we end up with a mess.
Comments
And another thing I was thinking about is handling of comments. When I created my example serviceList I added lots of comments (of course this doesn't make the list itself more easily readable, but was good to have at least for the example.)
Are there any wishes from your side, whether comments should be added, not added or added only in a specific format (e.g. just short notes, etc.)) so we don't end up with lots of specifications looking totally different - or are those "irrelevant" as they are ignored anyway?


Reordering the tutorials?

Looking at the tutorials, the ordering looks a bit random. Perhaps we should reorder the tutorials, roughly in order from least to most advanced, to make it easier to understand them? I'd suggest something like:

Introduction
Linear regression
Logistic regression
Multinomial regression
Poisson regression
Gaussian mixture model
Infinite mixture model
Variational inference
Bayesian neural networks
Bayesian differential equations
Hidden Markov models

So the order goes from easy to difficult; this order also keeps generalized models together with their more specific instances (e.g. logistic regression is followed by multinomial regression).
I can make the PR if you guys are interested.

Yep, I like that ordering. Currently they're just organized in the order that they were written, which is not super great as you've noted.
The big issue here is that moving everything around will break all the old links to these tutorials, which isn't super great. I'm kind of tilting towards not rearranging them for just that reason, but I'm very open to being persuaded.

Yep, I like that ordering. Currently they're just organized in the order that they were written, which is not super great as you've noted.
The big issue here is that moving everything around will break all the old links to these tutorials, which isn't super great. I'm kind of tilting towards not rearranging them for just that reason, but I'm very open to being persuaded.

Perhaps we could redirect the old links to the new links, and remove the numbers from the link so this won't be a problem in the future?

Sure, why not? I'm open to a PR.

Sure, why not? I'm open to a PR.

Cool, I'll make a PR here; do you know who usually handles the JS and such for the website and could get a redirect link working? I don't know how to do web dev.

Cool, I'll make a PR here; do you know who usually handles the JS and such for the website and could get a redirect link working?

You need to figure out how people do it in Franklin.jl. Redirects are a native HTML construct, by the way, not Javascript per se.

Oh sorry. Turing isn’t using Franklin. Then, just place the correct redirect files at the right places, I guess

Oh sorry. Turing isn’t using Franklin. Then, just place the correct redirect files at the right places, I guess

What would those places be?

Big +1 on reordering the tutorials - as a newcomer, the current order is indeed confusing. Worse than that, it starts with subjects that sound daunting to me, killing the "let's start at the beginning and learn this stuff!" feeling/motivation right away.

@yebai It just need updating _quarto.yml, Is it fine to rename them as you were saying no to restructure the file organization previously?
I can either rename the files and updated _quarto.yml or else reorganize whole file structure of docs like this:
All this in docs repo, using-turing, contributing & for-developers folders in root directory:
├───using-turing
│   ├───getting-started
│   ├───more...
├───for-developers
│   ├───abstractmcmc-turing
│   ├───more...
├───tutorials
│   ├───bayesian-differential-equations
│   ├───more...
├───contributing
│   ├───contributing-guide
│   └───contributing-style-guide

by renumbering and updating quarto.yml, If I am right?

Introduction

Linear regression

Logistic regression

Multinomial regression

Poisson regression

Gaussian mixture model

Infinite mixture model

Variational inference

Bayesian neural networks

Bayesian differential equations

Hidden Markov models

Is this sequence correct?
and what about these:

probabilistic-pca
gplvm
seasonal-time-series
minituring
gaussian-processes

and also get-started/using-turing section's sequence is correct? also for developers section!

@yebai It will of great help if you provide the sequence!


fix flaky tests

This can be a list of flaky tests that need to get fixed at some point.
https://github.com/denoland/fresh/pull/1715 failed with the following problems:

https://github.com/denoland/fresh/actions/runs/6021026772/job/16333211788?pr=1715

Resolves routes with non-latin characters => ./tests/main_test.ts:890:6
error: Leaking resources:
  - A fetch response body (rid 109) was created before the test started, but was consumed during the test. Do not close resources in a test that were not created during that test.

https://github.com/denoland/fresh/actions/runs/6021026772/job/16333211975?pr=1715

revive island in lazy server rendered children conditionally => ./tests/islands_test.ts:498:6
error: Leaking async ops:
  - 1 async operation to op_read was started before this test, but was completed during the test. Async operations should not complete in a test if they were not started in that test.

@lucacasonato was able to trace the warnings down to a bug in the test ops sanitization code. The fix is not yet ready to land, but hopefully soon. https://github.com/denoland/deno/pull/20501
Apart from that it seems like puppeteer is leaking timers. One idea there is to extract puppeteer tests into separate files and disable ops sanitization there. Disabling this in a file which has other tests trips up the leak detection.

Resolved by https://github.com/denoland/fresh/pull/1786


refactor collision detection code

there are potentially defects with this code as it stands now. On the first level it is easy to 'catch' on the bottom edge, becoming unable to turn as expected


Added minLength to prevent empty properties

What would the difference in meaning be between submitting e.g. demandLevel as null and not submitting it? In general it's best to either mandate and allow null, or leave the property optional and prohibit null (as up to now) unless there's a difference in meaning between the two.

With oneOf type array, we have to push multiple schema I think,
one with both properties, one with only one of them and the reverse.
If I remove one of the property, it actually invalid the file.

It's strange cause those properties aren't in "required"
I'll test a few more and come back, don't pull right now.

Ok, tested, was testing wrong actually !
Reverted and ok to me.


Add Section 1 Dialog Sounds

Added initial section 1 dialog sounds
Created Sounds.ts and moved getSoundDuration there
GetSoundDuration seems to return 0 pretty often for some reason. Storing the sound durations in an autogenerated lookup because of that.

Might have fixed https://github.com/ModDota/dota-tutorial/issues/146 in here too, let me know if you get bugs with panning after this one still.


Allow user to specify time window CriMS uses to derive intensity data.

Remember that CRIMS calibration data contains Covid-19 and its impact on crime
Likely solved by allowing user to specify time window (i.e. 2017-2019) for calibration data

for the current implementation we will just hard-code to the 2 years immediately preceding covid (4/2018-3/2020) which aligns nicely with the (quarterly) ONS count data.


api: add socket buffer limit field in ClientTrafficPolicy and BackendTrafficPolicy

What this PR does / why we need it:
Allow user to configure socket buffer limit for both incoming socket and outgoing socket to control memory consumed by Envoy, especially useful for user case of long-lived L4 proxy for massive connections.
It's different from existing connection buffer limit api which is used to control user space memory consumption, and this new api is used to control kernel memory consumption.
Which issue(s) this PR fixes:

Fixes #2670

/retest

/retest

/retest

/retest

/retest


How can we use ops.reshape when the new_shape is obtained from ops.shape?

It failed when using ops.reshape in model construction with dynamic shapes.
When using tf.keras, the same script ran without any issue.
import keras_core
import tensorflow as tf

"""
keras_core: Failed
"""

inputs = keras_core.layers.Input(shape=(None, None, 3))
b, h, w, c = keras_core.ops.shape(inputs)
x = keras_core.ops.reshape(inputs, (b, h, w, c, 1))
model = keras_core.models.Model(inputs=inputs, outputs=x)

x = keras_core.ops.random.uniform(shape=(1, 28, 28, 3))
y = model(x)
print(y.shape)

"""
tf.keras: Successed
"""

inputs = tf.keras.layers.Input(shape=(None, None, 3))
b, h, w, c = tf.shape(inputs)
x = tf.reshape(inputs, (b, h, w, c, 1))
model = tf.keras.models.Model(inputs=inputs, outputs=x)

x = tf.random.uniform(shape=(1, 28, 28, 3))
y = model(x)
print(y.shape)

I believe there are two issues:

ops.shape returns tuple of ints or None when using symbolic tensor
new_shape in ops.reshape might contain None (and this caused error)

The solution might be using tf.shape for ops.reshape, but I'm not sure how to implement the fix.
Colab: https://colab.research.google.com/drive/1UYaIxAqmHSUiR4nJyULya8w14Ha19rhB?usp=sharing

Just came up with a workaround after posing:
"""
Workaround
"""
class CustomReshape(layers.Layer):
    def __init__(self, name=None):
        super().__init__(name=name)

    def compute_output_shape(self, input_shape):
        return (*input_shape, 1)

    def call(self, x):
        b, h, w, c = tf.shape(x)
        return tf.reshape(x, (b, h, w, c, 1))

inputs = keras_core.layers.Input(shape=(None, None, 3))
x = CustomReshape()(inputs)
model = keras_core.models.Model(inputs=inputs, outputs=x)

x = tf.random.uniform(shape=(1, 28, 28, 3))
y = model(x)
print(y.shape)

The custom layer is the right idea.
ops.reshape works with ops.shape, but only with actual tensors -- not just symbolic tensors, that is to say Input objects. That means that a tensor's shape can be always be known inside the call method of a layer, which is why querying ops.shape and calling reshape inside call() works fine.
The reason it worked with tf.keras is that in tf.keras, an Input is backed by a TF tensor. This is not the case with Keras Core, Input is just a standalone Python object and its shape may contain None entries.

Thanks for the detailed clarification.
I provide the following backend-agnostic workaround in case anyone encounters this issue
import keras_core
from keras_core import layers

"""
Workaround: ops.reshape works with ops.shape with actual tensors
"""

class CustomReshape(layers.Layer):
    def __init__(self, name=None):
        super().__init__(name=name)

    def compute_output_shape(self, input_shape):
        return (*input_shape, 1)

    def call(self, x):
        b, h, w, c = keras_core.ops.shape(x)
        return keras_core.ops.reshape(x, (b, h, w, c, 1))

inputs = keras_core.layers.Input(shape=(None, None, 3))
x = CustomReshape()(inputs)
model = keras_core.models.Model(inputs=inputs, outputs=x)

x = keras_core.ops.random.uniform(shape=(1, 28, 28, 3))
y = model(x)
print(y.shape)


Screw MSIE 8/9 support.

Hello,
I would like to suggest to be pragmatic here and screw support for these rather obsolete browsers.
Mainly, to prevent the bloated code, and also i think it will make further maintainability with the Node.js events API easier in the long run.
Thanks for consideration.

hmm, events is pretty heavily depended upon as a browser version of node's events, most notably by browserify and webpack; dropping IE8 support here would mean that browserify and webpack bundles that use the core events module would also no longer work in IE8 (and other ES3 browsers).

That's quite right... however, how many would be affected? all products i'm familiar with have dropped support for it years ago already.

IE9 supports most of ES5 so we can probably support that fairly easily. I'm OK with doing this otherwise.


Not able to import vartests even after installation

I have installed vartests using various methods. One among them is from Jupyter, the other is from pip install vartests. I am able to see vartests directory in site-packages, but when I try importing the package I am unable to do so. I have tried all the methods that are found in the web to install and debug but to no avail. I just wanted to understand if the package is good to be used.

Hi @deeppradhan, Sorry for the inconvinient. Actually I`ve just rearraged the package to be installed correctly. Hope works for you.

Hi
Can I just pip install the package in that case?
Regards,
Deep
On Mon, 26 Sep 2022 at 5:36 AM, RAFAEL RODRIGUES @.***>
wrote:

Closed #3 https://github.com/rafa-rod/vartests/issues/3 as completed.
—
Reply to this email directly, view it on GitHub
https://github.com/rafa-rod/vartests/issues/3#event-7454272335, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AB4XS27YJDNC2XUAHN4DB53WADSJZANCNFSM6AAAAAAQVA53GQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Deep

Yes. To make sure you are installing tbe right package, I recommend pip uninstall vartests first, then pip install vartests.the version should be <IP_ADDRESS>Enviado do meu iPhoneEm 25 de set. de 2022, à(s) 21:45, Deep Narayan Pradhan @.***> escreveu:
Hi
Can I just pip install the package in that case?
Regards,
Deep
On Mon, 26 Sep 2022 at 5:36 AM, RAFAEL RODRIGUES @.***>
wrote:

Closed #3 https://github.com/rafa-rod/vartests/issues/3 as completed.
—
Reply to this email directly, view it on GitHub
https://github.com/rafa-rod/vartests/issues/3#event-7454272335, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AB4XS27YJDNC2XUAHN4DB53WADSJZANCNFSM6AAAAAAQVA53GQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Deep
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>

Thank You so much!
Appreciate it
I’ll try and let you know if I face any issues.
Regards,
Deep
On Mon, 26 Sep 2022 at 6:31 AM, RAFAEL RODRIGUES @.***>
wrote:

Yes. To make sure you are installing tbe right package, I recommend pip
uninstall vartests first, then pip install vartests.the version should be
<IP_ADDRESS>Enviado do meu iPhoneEm 25 de set. de 2022, à(s) 21:45, Deep Narayan
Pradhan @.***> escreveu:
Hi
Can I just pip install the package in that case?
Regards,
Deep
On Mon, 26 Sep 2022 at 5:36 AM, RAFAEL RODRIGUES @.***>
wrote:

Closed #3 https://github.com/rafa-rod/vartests/issues/3 as completed.
—
Reply to this email directly, view it on GitHub
https://github.com/rafa-rod/vartests/issues/3#event-7454272335, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AB4XS27YJDNC2XUAHN4DB53WADSJZANCNFSM6AAAAAAQVA53GQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Deep
—Reply to this email directly, view it on GitHub, or unsubscribe.You are
receiving this because you modified the open/close state.Message ID:
@.***>
—
Reply to this email directly, view it on GitHub
https://github.com/rafa-rod/vartests/issues/3#issuecomment-1257337923,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB4XS23BFZPZYAWJLGLZ7FTWADYW7ANCNFSM6AAAAAAQVA53GQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Deep

This upgrade works like magic. Thanks a lot!
On Mon, 26 Sep 2022 at 7:53 AM, Deep Pradhan @.***> wrote:

Thank You so much!
Appreciate it
I’ll try and let you know if I face any issues.
Regards,
Deep
On Mon, 26 Sep 2022 at 6:31 AM, RAFAEL RODRIGUES @.***>
wrote:

Yes. To make sure you are installing tbe right package, I recommend pip
uninstall vartests first, then pip install vartests.the version should be
<IP_ADDRESS>Enviado do meu iPhoneEm 25 de set. de 2022, à(s) 21:45, Deep Narayan
Pradhan @.***> escreveu:
Hi
Can I just pip install the package in that case?
Regards,
Deep
On Mon, 26 Sep 2022 at 5:36 AM, RAFAEL RODRIGUES @.***>
wrote:

Closed #3 https://github.com/rafa-rod/vartests/issues/3 as completed.
—
Reply to this email directly, view it on GitHub
https://github.com/rafa-rod/vartests/issues/3#event-7454272335, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AB4XS27YJDNC2XUAHN4DB53WADSJZANCNFSM6AAAAAAQVA53GQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Deep
—Reply to this email directly, view it on GitHub, or unsubscribe.You are
receiving this because you modified the open/close state.Message ID:
@.***>
—
Reply to this email directly, view it on GitHub
https://github.com/rafa-rod/vartests/issues/3#issuecomment-1257337923,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB4XS23BFZPZYAWJLGLZ7FTWADYW7ANCNFSM6AAAAAAQVA53GQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Deep

--
Regards,
Deep


MembershipReboot, MVC Authenticaticion samples don't build :(

first two I tried...

And what are the errors?

they were package restore errors due to file path too long as you probably guessed .. I got around it by running the samples from the drive root instead of C:\Users\User\Source\Workspaces


Include updated image in commit message

Currently there is no(?) support to include the updated image/tag in the commit message.
This does not provide a very useful commit log when the automation is used.
The default, "Update from image update automation" should also include the resource being updated and to which image or tag it has updated to.
Expected (suggestion):
For an HelmRelease: The default commit message should be in the form of " updated to ", e.g "service1 updated to 1.2.3-a6b8c6c563b5f3db2ceb1ee9d548acd2abb6baaf-12345"
Should be supported in both the default commit message as well as in a custom messageTemplate
Desired:
Possible to configure a global commit message template for the entire cluster

Besides the commit message, the image-automation-controller should issue an event with all the updated images. This way using notification-controller, users can see the info on Slack/MS Teams/etc.

Currently there is no(?) support to include the updated image/tag in the commit message.

I agree this would make it much more useful!
In general, more than one image and more than one file / object can be updated in a commit. But it is possible to list them out or branch depending on how many there are, in a text template.

MBOI: #119 (see what you think of the template data defined there)


Run GRPC server over TLS

Release note:

GRPC service is now served over TLS

/approve cancel

@wattli @lizan PTAL


fix(typescript): the prop type doesn't need to include the default value

the whole point is that the default value is never provided
plus, it was generating invalid TypeScript in messages.d.ts
Related: https://github.com/phenomejs/phenome/issues/11

Great, thanks!


Document and refine the /health endpoint

I see that credhub is exposing a /health endpoint (which I could not yet find in the API documentation). This endpoint does not seem to check dependent resources health https://github.com/cloudfoundry-incubator/credhub/blob/c7aa6f0a51dbdc0ac48a5442b889adc9edb51db1/src/main/java/org/cloudfoundry/credhub/controller/v1/HealthController.java#L19
It seems the history of spring actuator code-like reuse indeed included related datasource health check  https://github.com/cloudfoundry-incubator/credhub/commit/4702ca127749be9b6687b104ddc7cb6f1ea199ba
While external systems such as https://github.com/orange-cloudfoundry/credhub_exporter can monitor credhub health using find-credentials or get-credentials endpoint, the /health endpoint seems appealing (for performance and dependency coverage reasons).
I wonder whether direct usage of spring actuator was recently considered (as I could not find mention of it in the credhub pivotal tracker), and may address potential (performance?) issues that occured in the past.
/CC @psycofdj psycofdj

What version of the credhub server you are using?

2.0.2

@gberche-orange What do you mean by:

This endpoint does not seem to check dependent resources health

@athornton2012 As an example, I mean that the /health endpoint does not check that the underlying resources such as the credhub store or configured HSM is also available. In other words, it may respond UP while all credhub requests may fail

@gberche-orange makes sense, perhaps it would be worthwhile for us to improve our /health endpoint. @crawsible any thoughts/hesitations?

I'm open to being convinced otherwise but it doesn't seem obvious to me that our health endpoint should be doing anything more than reporting on the proper functioning of the server itself. The health endpoint primarily exists for BOSH and the post-start script. My expectation is that UAA, a dependent DB, an HSM etc. are also being monitored if they're important responsibilities.
As I said I'm open to being convinced otherwise though. Could you tell me more about your use-case, and in particular what value this would bring you over and above simply monitoring the dependent services as well?

@gberche-orange bumping on this

@crawsible @athornton2012
Sorry for late follow up. While the dependent services would indeed be monitored separated, it seems important to me to have credhub report failures when network partitions prevents credhub to access to UAA/DB/HSM, which results in credhub being unable to provide its service.
Springboot actuator automates this practice for various dependent resources, see https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html#_auto_configured_healthindicators

@gberche-orange - we recently updated the /health endpoint in CredHub as a part of this story: https://www.pivotaltracker.com/story/show/158726819. CredHub now uses Boot healthcheck functionality. Is this the same functionality that you are referencing? If so, does it resolve your ask here? Thank you!

thanks @ankeesler
I can now see the /health endpoint being documented at http://credhub-api.cfapps.io/version/2.1/#health although response body does not seem described. Might be worth adding a link to https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html#_writing_custom_healthindicators   or https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/actuate/health/Status.html
Besides, I believe the health endpoint would benefit from reporting as down when it is unable to access its UAA/DB/HSM.
I could not see this being described in the story acceptance criteria.
I would assume that spring boot default health indicator would be enabled by default
https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html#_auto_configured_healthindicators
including the DataSourceHealthIndicator that would test access to the credhub db by default.
It might make sense to add a uaa health indicator to test connectivity to the uaa server.

@gberche-orange There are other ways to check the health of UAA/DB/HSM, like hitting the UAA health endpoint, attempting to store something in the DB, and attempting to encrypt something with the HSM. That would be a lot of work to check the health of CredHub but this feature request doesn't seem like a top priority for our team at the moment. We are going to close this issue but will reopen if we receive similar requests.


Fixing Chunking issues with codec

This fixes #460.
The PR add the option that defaults for max chunk sizes of 500MB or max_chunk_size_bytes in mantis.cli.utils.create_empty_hcs().
Additionally the chunk size is set to Optional=None so that if None, the default chunks are 500MBs.

I looked closer to the issue and I realized that the issue is during the saving of the transfer functions. Here you would have to create_empty_zarr() or add the position.create_zeros() beforehand rather than allocating and saving the 'fov' directly into zarr here @talonchandler


Library file locations

I don't expect to get a reply to this because there seems to be no support for CADMesh but I wanted to raise it as an issue:
CADMesh for some reason decide to put libcadmesh.so in a different directory from the other library files that it uses (libassimp.so and libtet.so). I don't know if this is deliberate but it seems odd. Is there an error somewhere in CMake?
I note, again, that the instructions for installing CADMesh don't seem to work on many systems.

Please try the latest version. Version 2. There is no longer a requirement for assimp and tegen if you are only interested in loading OBJ, STL or PLY files.


[Feature] OrdinalScale class

Create class representing Ordinal Scale.
name: OrdinalScale
init params:

levels (list-like) / example levels: ["basic", "medium", "hard"]
orders (optional, list-like, integers, every number must be non-zero and positive) / example orders: [2, 4, 6]

attributes:

levels (the same as init param)
orders - if orders parameter is not given then this attribute must be defined as range from 1 to len(levels) + 1.

where?

in src.core.py

I'll take this

In the OrdinalScale class, which arguments are the and info funcitons supposed to take respectively to make them attributes to statistics and distribution respectively? Unless you mean setting these functions equal to class functions

Hi @imrannasrullah
I see what you meant! I am sorry for the confusion; using those functions is impossible. The class doesn't have access to the required data. Thus, I've redefined the first iteration, now it's extremely simple. It will grow with time; now we need a placeholder that stores information about ordinal scale ranges (in real-world settings, you could have a form with descriptive levels, as in the example).

@imrannasrullah I've assigned you to this task. Please, create your own branch from the main, and when you are done, then push PR into main. The second thing, please, create boilerplate docs for this class following numpy style docs (https://github.com/hs3city/ordinal-scale-stats-py/blob/main/docs/dev-docstrings.md) - only parameters and attributes sections, don't bother with descriptions, I'll do it later.


Fix last change to work in all locales

the grep for "fingerprint" only works in C locale, not if you
use a different locale.

+1

+1

sure that LC is visible inside the grep subshell?

@aplanas you never trust anything, eh? :)

root@crowbar:~ # cat <(LC_ALL=C gpg --with-fingerprint /srv/tftpboot/suse-12.1/aarch64/repos/Cloud/content.key)
pub  2048R/F99041E1 2013-08-12 [expires: 2018-09-21]
      Key fingerprint = BBB0 A65B 9E2D 41F4 5575  4979 4726 6566 F990 41E1
uid                            Devel:Cloud OBS Project<EMAIL_ADDRESS>root@crowbar:~ # cat <(LC_ALL=es_ES gpg --with-fingerprint /srv/tftpboot/suse-12.1/aarch64/repos/Cloud/content.key)
pub  2048R/F99041E1 2013-08-12 [caduca: 2018-09-21]
      Huella de clave = BBB0 A65B 9E2D 41F4 5575  4979 4726 6566 F990 41E1
uid                            Devel:Cloud OBS Project<EMAIL_ADDRESS>

ah ok, we want only to influence gpg not the grep
sorry for the noise: +1

+1
Of course a better alternative would be:
read -a fingerprint < <(LC_ALL=es_ES gpg --with-fingerprint ${key_file} | grep clava | awk -F= '{print $2}')

@nicolasbock that requires gpg2-lang to be always installed :)


I've installed Stripheaders on my Webserver w/ IIS 8.5 but it isn't working.

I can see the configuration and view it in the Configuration Editor and it looks good but its not working.

Just went through and manually registered the module, it appears to be working afterword.


5.16 sfContactTraceRecord Lambda updates fields in Salesforce that do not exist in the 5.16 Salesforce package

The amazonconnect__DisconnectReason__c field does not exist in the 5.16 version of the Salesforce managed package.
Reproduction Steps:

Install 5.16 Salesforce managed package to Salesforce with CTR Import parameter
Install 5.16 Salesforce lambda package from SAR
Call with a flow that sets the postcallCTRImportEnabled attribute
Check logs on sfContactTraceRecord lambda

Stack Trace Below:
[ERROR] Exception: INVALID_FIELD: No such column 'amazonconnect__DisconnectReason__c' on sobject of type amazonconnect__AC_ContactTraceRecord__c Traceback (most recent call last):   File "/var/task/sfContactTraceRecord.py", line 45, in lambda_handler     raise e   File "/var/task/sfContactTraceRecord.py", line 40, in lambda_handler     process_ctr_record(event['record'])   File "/var/task/sfContactTraceRecord.py", line 55, in process_ctr_record     create_ctr_record(record_obj)   File "/var/task/sfContactTraceRecord.py", line 141, in create_ctr_record     sf.update_by_external(objectnamespace + "AC_ContactTraceRecord__c", objectnamespace + 'ContactId__c', ctr['ContactId'], sf_request)   File "/var/task/salesforce.py", line 114, in update_by_external     self.makeRequest(self.request.patch, **{"url": url, "data": data})   File "/var/task/salesforce.py", line 183, in makeRequest     return requestMethod(**kwargs, headers=self.headers)   File "/var/task/salesforce.py", line 227, in patch     return __check_resp__(r)   File "/var/task/salesforce.py", line 253, in __check_resp__     raise Exception(msg)

Commenting line 106 of the sfContactTraceRecord.py file is fixing the issue


Firebase Login results in Thread 1 Bad Instruction: unexpectedly found nil

Hi,
so when logging in with a correct username/password combination, my App crashes, with the following error:

fatal error: unexpectedly found nil while unwrapping an Optional value
(lldb)

Furthermore, there is a Thread 1 at the following line in the didTapEmailLogin method:
self.ref.child("users").child(user.uid).observeSingleEvent(of: .value, with: { (snapshot) in

Thread 1: EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)

This could just be me, I am not sure, maybe someone here has a similar problem.
I am using the latest Swift 3 code by the way.

Can you post the full stack trace here?

Looks like the line of code you're referencing no longer exists in the project, so I'm going to close this issue. Re-open if it's still an issue.


Support azure-eventhubs

I have not found any evidence that Azurite supports azure-eventhubs. Therefore I assume it does not. If so, would it please be possible to add support for azure-eventhubs?

@ppalaga
Do you mean you would like to have Azurite support azure-eventhubs API?
Or you mean you find Azurite doesn't work on storage API when collaboration with Eventhub?

Do you mean you would like to have Azurite support azure-eventhubs API?

Yes, this one, please.

@ppalaga
Azurite is an open source Azure Storage API compatible server (emulator). So currently we only target to support Azure Storage APIs.
I will try to help you contact the eventhub team, and inform them the request.


add(tools): Windi CSS

Name
Link

Windi CSS
https://github.com/windicss/windicss

Windi CSS is a Tailwind CSS compatible library that focuses on compilation speed.

[X] My item is in the right category
[X] My item is logically grouped below similar items
[X] My item's name and description respects the conventions of the list
[X] My item is awesome
[X] I have read and followed the contribution guidelines

Thank you! Can't believe I forgot to add it before, haha.


Auto-generate fake REST API from swagger

I know this would be a huge task and might be better suited as a third-party tool. But I think what would be really cool is if there was a way to auto-generate Janus config from swagger. Many REST APIs use swagger for documentation, and there are a number of tools available to automatically generate client libraries, for example:

https://www.npmjs.com/package/swagger-models-to-react-proptypes
https://github.com/swagger-api/swagger-codegen

So it should be doable to auto-generate janus config from a swagger documented API. It's just an idea but I think it would be cool.

Great idea.  Added this as a feature for next milestone

👍 👍 👍

that would be awesome. Already have my api declared in swagger or upcoming openapi. 👍 👍 👍


engine add command fails with Can't locate object method "config"

Call and error:
vagrant@Debian-jessie-amd64-netboot:~/veure$ sqitch engine add pg veure_test
Can't locate object method "config" via package "App::Sqitch::Command::engine" at /home/vagrant/perl5/perlbrew/perls/perl-5.18.4/lib/site_perl/5.18.4/App/Sqitch/Command/engine.pm line 88.

sqitch.conf:
[core]
	engine = pg
	top_dir = sqitch

Sqitch version:
vagrant@Debian-jessie-amd64-netboot:~/veure$ perl -MApp::Sqitch -e 'print App::Sqitch->VERSION'
0.9993

Not sure if i'm doing something wrong, but this looks like a sqitch bug.
Edit: Forgot to mention: Calling it without any target just works, but results in:
[engine "pg"]
	target = db:pg:

Right, already fixed in f4d75dd. Thanks!

I'd like to ask you to consider a small request: Could you leave patched, but unreleased issues open and tag them as patched? I did search your issues, but did not think to search the closed ones as well, so i thought this had not been reported yet.
Also, any idea on when a release is planned?

Well, when I pull a commit from a pull request, GitHub immediately closes it.
Dunno when I'll do another release, not actively doing Sqitch stuff at the mo. Probably later this month or early next month.

Thanks for the estimate. :)


Fixes

Hello @swapsha96! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file mirapy/fitting/tests/test_optimizers.py:

Line 32:70: W292 no newline at end of file


Add is_syncing flag

PR Description
Add the is_syncing field to /eth/v1/node/syncing endpoint as per https://github.com/ethereum/eth2.0-APIs/pull/121/
Fixed Issue(s)
fixes #3528
Documentation

[x] I thought about documentation and added the documentation label to this PR if updates are required.

Changelog

[x] I thought about adding a changelog entry, and added one if I deemed necessary.

this doesn't change any tests? that's odd...

Yeah that surprised me a bit too.  Turns out there is an integration test, but it was only checking specific fields individually and not comparing the whole object so it kept passing.  Updated it.


add multiline comments

''' does not work
generate "Unexpected token: Constant" error

Duplicate of #1320.


detect if debug build from java without any obj-c code

i have been working with MOE for a few months and managed to write a whole game in java using the apple API, and i love the fact ive not had to learn any new language syntax, and ive been able to do everything i can dream of in java, everything except one thing, detect if this is a debug build, ive been told that this is really simple:
"You could write a small function in ObjC that returns different value with "#ifdef DEBUG", then call this from your Java code."
but considering i have never written any ObjC, have no idea where to put the file, what to put into the file, or how to then call it from java, things ive never needed to do so far, i would actually need to learn quite a lot to do this simple task.
also swift and kotlin has "Platform.isDebugBinary" so java should have a method too!
there are many workaround for this, but its always going to be easier to just have a nice method to call

another thing that could also be really useful and not really possible to currently do without obj-c code is having a method that returns if the app is running on a iOS Simulator.
in swift you can do
#if targetEnvironment(simulator)
but as java does not have the #if command, it would be good to be able to call a normal method for
isTargetSimulator

another thing that could also be really useful and not really possible to currently do without obj-c code is having a method that returns if the app is running on a iOS Simulator.
in swift you can do
#if targetEnvironment(simulator)
but as java does not have the #if command, it would be good to be able to call a normal method for
isTargetSimulator

Yes that's also in my mind. Actually this will be easier than checking if it's debug build, as simulator build will be linked to a different MOE framework, so this check can be baked inside the MOE framework, while the debug check need to happen when the app itself is built.

My current idea is splitting the current moevm function (https://github.com/multi-os-engine/moe-core/blob/ecca888c9b7c30a129ad943a06bdab88dd7f3128/moe.apple/moe.core.native/moe.sdk/src/MOE.mm#L75) into two functions: moevm_release and moevm_debug, which will define a MOE_DEBUG environment variable before calling the real moevm function, i.e.,:
MOE.mm:
int _moevm(const int jargc, char* const* jargv) {
    // The existing moevm function
}

extern "C" int moevm_release(const int jargc, char* const* jargv) {
    setenv("MOE_DEBUG", "false", true);  // Set debug env variable to false
    return _moevm(jargc, jargv); // Call the actual moevm
}

extern "C" int moevm_debug(const int jargc, char* const* jargv) {
    setenv("MOE_DEBUG", "false", true);  // Set debug env variable to true
    return _moevm(jargc, jargv); // Call the actual moevm
}

Then in the header (https://github.com/multi-os-engine/moe-core/blob/ecca888c9b7c30a129ad943a06bdab88dd7f3128/moe.apple/moe.core.native/moe.sdk/src/MOE.h) we define the macro moevm to moevm_release or moevm_debug based on whether it's debug build or not:
MOE.h
#ifdef DEBUG
EXTERN int moevm_debug(const int jargc, char *const *jargv);
#define moevm moevm_debug
#else
EXTERN int moevm_release(const int jargc, char *const *jargv);
#define moevm moevm_release
#endif

This is the easiest way I could think of without the need of updating any of the user project files.
However since MOE 1.9 requires an update to the xcode project file, this is not really a problem. We can of course add a function like this to the main.cpp file (or a separate file):
__attribute__((visibility("default"))) int is_debug() {
#ifdef DEBUG
    return 1;
#else
    return 0;
#endif
}

then call this function from java.

i thought you were going to put that thing you mentioned 'ObjC that returns different value with "#ifdef DEBUG" ' into the existing generated obj-c file moe-main-interfaces.m thats showing up as part of the project, i dont really understand what the file does.
regenerating project files is totally fine, especially if it makes the code simpler
and if java having a method or env variable are both perfect

Another solution is to generate a config file that will be bundled into the app during the build process, which contains the debug flag, then read this file in Java. The benefits is we can add more build-time configs in it later, but the downsides are:

More codes
Require updating project file
Could lead to inconsistency between Java & native codes (for example if you removed the DEBUG macro from the Xcode debug config, which will only affect the native codes but not the Java part)

i dont really understand what the file does.

This allows you to use interface builder with the ViewControllers you created in Java.

Then in Java we could have a method that reads this environment variable to check if it's debug or not.
public static boolean isDebug() {
    return "true".equals(System.getenv("MOE_DEBUG"));
}

Have you tested this? I remember that in the standard JRE System.getenv() sometimes works a bit strange and isn't able to fetch a env variable that was updated with setenv after the process was started. So maybe a java native method is needed that runs getenv.

Then in Java we could have a method that reads this environment variable to check if it's debug or not.
public static boolean isDebug() {
    return "true".equals(System.getenv("MOE_DEBUG"));
}

Have you tested this? I remember that in the standard JRE System.getenv() sometimes works a bit strange and isn't able to fetch a env variable that was updated with setenv after the process was started. So maybe a java native method is needed that runs getenv.

The environment variable will be set BEFORE the JVM is started so it can work. We are doing similar thing already.

DEBUG, IS_SIMULATOR flags are added to class org.moe.MOE


chmod required in travis build

This is not the case in examples like caffe....

Closing this, assuming it's long forgotten, Travis is working fine.


Limit Trading Feature

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Feature

**What is the new behavior (if this is a feature change)?
Possibility to configure the maximum amount of fiat currency available to gekko to trade with.
For example if you have 1 BTC and 100 USD in an exchange account which gekko is performing BTC/USD trading on, you can set currencyAmountAvailableToTrade in your config file to something like 50 USD. If a buy comes in, only 50 USD is used and further buys are prevented. If sells occur, all USD proceeds get added to the available tradable balance ready for the next BUY order. In this way, the other 50 USD of your exchange balance remains untouched.

I don't use the web version and haven't looked into it's code so it is not compatible with it.

Other information:
Cleaned up and resolved conflicts of PR #1492

The only thing I don't understand is where you set up the amount or percentage that will be negotiated.

See sample-config.js
The is passed to portfolioManager near line 59.

This is the single most requested feature of Gekko, thanks for the PR.
A few notes on it:

I love your new added functionality where you track the amount already traded on these trades. However I propose we add it in a way where this code is always used (not just when this feature is enabled), mostly to track bugs and such.
When you are bailing out because the amount is to low you are returning without calling proper callbacks (such as checkOrder).
you are calling checkOrder after every individual buy/sell, not when they are all done (this is also a bug in the current implementation, quite possibly the bug described in #1852).
We need to think about implications: this calculates the max amount to buy/sell based on current rates, this means that if you trade USD/BTC and the price goes up and down you might lose more than 100 USD over a timespan. Is this really how we want to implement this?

Given this in mind I propose to create something a bit more clear and subtle: a new class that is responsible for doing a single trade (let's call him broker or execution strategy or so), it needs to:

Figure out if there is a limit on the size of the trade we can do.
If we can't do any trade (low funds) don't do anything.
If we can do a trade figure out what kind of trade we aim to do (sell X USD) and trigger an event (see #1850).
Try to buy sell according to current "limit order do not cross spread" strategy (explained here).
only when fully completed figure out trade statistics (average execution rate, amount that was traded, etc).
only after this upstream a trade event with the statistics.

The reason I want to pull this out is so that in the future we can have different execution strategies: the current one tries to get the best price - at the cost of execution speed. Once it's split out we can easily create another one that will do a market order instead. Or one that tries to split the order out into multiple small ones (bigger traders need ways to take pressure of the market for example).

@werkkrew @Ali1 what are your thoughts?

@askmike

I love your new added functionality where you track the amount already traded on these trades. However I propose we add it in a way where this code is always used (not just when this feature is enabled), mostly to track bugs and such

Thank you. This code always runs, even without limit trading feature enabled, this is what you will get:
2018-01-27 02:56:12 (INFO):	BUY order was not (fully) filled, cancelling and creating new order
2018-01-27 02:56:12 (INFO):	So far, traded 0.01787247 ETH for approx 15.2437871124 EUR Approx average price: 852.92 <-- new
2018-01-27 02:56:12 (INFO):	Attempting to BUY 0.8334388630927254 ETH for<PHONE_NUMBER>343464 EUR price: 852.37
2018-01-27 02:57:13 (INFO):	BUY was successful after 12 tries, trading 0.01787247 ETH for approx 15.2437871124 EUR Approx average price: 852.92 <-- improved

When you are bailing out because the amount is to low you are returning without calling proper callbacks (such as checkOrder).

I haven't changed the bail out procedure when the balance is too low (i.e. 0 or lower than exchange minimum). If you mean trading limit is set to 0, then I'm using the same bail out procedure which is for there not to be any callback.

you are calling checkOrder after every individual buy/sell, not when they are all done (this is also a bug in the current implementation, quite possibly the bug described in #1852).

Sorry not sure about this, I didn't change any callbacks. I didn't actually play a lot with orders that don't fill completely except for adding additional logging. My code was more related to whether the trade should go ahead and changing the amount.

We need to think about implications: this calculates the max amount to buy/sell based on current rates, this means that if you trade USD/BTC and the price goes up and down you might lose more than 100 USD over a timespan. Is this really how we want to implement this?

Again not sure about what you are saying here. Do you mean current rates at initiation (in which case, the use case is to set a maximum USD rate so that multiple bots can share the pot of USD) or current rae at each trade (in which case I think the way I've implemented it makes sense). Let me know.

Given this in mind I propose to create something a bit more clear and subtle: a new class that is responsible for doing a single trade (let's call him broker or execution strategy or so), it needs to:

This is a fantastic idea. I don't think I'm the right person to do it since I've learnt node for this PR, I've come from different development languages - I'm still trying to get my head around things. I think what I have added seems like a lot, but it's not really - a lot of my code is extra logging which I thought might be useful. Perhaps the next step, through a further PR, can be to package the trade method along with buy and sell methods and callbacks to a new class.
@greenbigfrog
There's a lot of inconsistency already.

@askmike

Given this in mind I propose to create something a bit more clear and subtle: a new class that is responsible for doing a single trade (let's call him broker or execution strategy or so)

I've been thinking about this for a few weeks, I like the idea of breaking it out into it's own class, that would provide much more flexibility. Brilliant, I added an issue for it here: https://github.com/askmike/gekko/issues/1942
@Ali1
This is great stuff, I was literally sitting down tonight to code this exact feature in, and I came across this. I hope we can get this merged in soon.
This is also a good point by @askmike :

We need to think about implications: this calculates the max amount to buy/sell based on current rates, this means that if you trade USD/BTC and the price goes up and down you might lose more than 100 USD over a timespan. Is this really how we want to implement this?

Another way to look at it, say you had $1000 total, with only $100 set as the amount you want to trade with, presumably you want $900 to be left untouched. Now you start trading with BTC, and it makes a trade which lost 20%, now there is $980 available to trade with. When it initiates a new trade, then it will take out another $100, now you only have $880 remaining. With a bad strategy left over a long period of time this could drain out your entire portfolio.
I quite prefer this approach @Ali1 has implemented over a "keep" approach, as for my own use case I would like to have multiple instances of Gekko running on different BTC pairs from one exchange account. So if I had 1BTC, I could have 10 bots trading all with a maximum of 0.1BTC, on different pairs / strategies.
I suggest a NET value, which is updated after each round trip to include the amount gained/lost. Then you could specify with a boolean option if you want to compound gains.
{
  currencyAmountAvailableToTrade: 0.1,
  compoundGains: true
}

Perhaps we can come up with some more simplified property keys. Thoughts??
BTW @Ali1 In your sample-config.js where you define the property currencyAmountAvailableToTrade = false should read currencyAmountAvailableToTrade: false ;)

is there an ETA for merging this PR?

@aapsoftware This PR has no merge conflicts so you could merge it into your local repository and you should be good to start testing it as-is (I'd suggest with very low amount of funds until you are sure it's working as you would expect). I'm sure any field testing / feedback would be beneficial.
It looks like this is the latest branch:
https://github.com/Ali1/gekko/commits/new-limit-trading

@aapsoftware
Would be useful for us if you tried it and go ahead and use Ali1/new-limit-trading
@ansonphong

Another way to look at it, say you had $1000 total portfolio, with only $100 set as the amount you want to trade with on this given strategy, presumably you want $900 to be left untouched. Now Gekko starts trading with BTC, and it makes a trade which loses 20%, now there is $980 remaining in your portfolio. When Gekko initiates a new trade, then it will take out another $100, now you only have $880 remaining. With a bad strategy left over a long period of time this could drain out your entire portfolio.

Yeah this is the easy way to do it but I haven't gone that way. The behaviour is supposed to work in a way where the most you could lose is the full "asset" (e.g. NANO) value stored in the exchange  plus the amount of currency (e.g. USD) that is set in the config. No matter what happens to the asset, how bad the strategy and how long the bot runs, there's no way that more USD can be lost than the value set in the config (if this works).

BTW @Ali1 In your sample-config.js where you define the property currencyAmountAvailableToTrade = false should read currencyAmountAvailableToTrade: false ;)

Fixed!
Getting a busier now so have stopped day trading, don't have an appetite to add more features at the moment, sorry. But shouldn't be difficult for more PRs to add extra features.

Perhaps we can come up with some more simplified property keys. Thoughts??

Suggestions welcome. Also I'm a little new to node. Would "let" in the trade method work for many of the trader class property keys?
Sorry getting quite busy so have stopped day trading for now. Don't have an appetite to add further features at the moment.

Might be useful to be able to alter the currency available while the bot is running, is this something that's possible? Would just need to change the currencyAmountAvailableToTrade property in portfolioManager and it will get picked up at the next trade.

@Ali1 Yes, let is the generally preferred way to scope variables in ES6. So let is for defining scoped variables, not object properties. It is generally a replacement for var, which orders definitions linearly, otherwise it is similar. For instance, you would use let when defining any variable in any scope, in place of var.

If more work is going to be done, perhaps the same mechanism here can be used to limit the amount of asset traded. Then the "keepasset" feature can be removed. Not that I think people would be using this feature as often.

@Ali1 I'll have a look at that, the Keep Asset functionality seems considerably simpler so it should be an issue.
@askmike Are you adverse to using ES6 syntax, or do you want to keep everything old school for the time being?

As I'm digging deeper into this, @askmike's idea about the Trade class is seeming more and more inevitable. I'm going to go ahead and do a prototype of the Trade class and submit a PR for that after some testing.

I took a look at the code; it seems to limit the amount the trader is allowed to trade with not the size of the trade. That is, you can limit your trading amount to 50% of your initial balance (that amount will grow if the trades are profitable) but it does not help if you want to trade a specific amount, always (i.e always place buy orders of 0.1 BTC).
It is helpful, but not quite what I was expecting.
I will give it a go and report back any issues I find.

@Ali1  limiting every trade to a fixed amount would not make much sense; what I was truly hoping for it the possibility to specify the amount I want to trade for each transaction; when you place an order it should be advice('buy/sell', price, amount)
On a different note, here's some field testing results:2018-02-22 04:44:43 (INFO):	In uptrend since
2018-02-22 04:49:23 (INFO):	In uptrend since 2 candle(s): @ 0.00330168
2018-02-22 04:49:23 (INFO):	BUY@  0.00330168  ( 0.0033099342 )
2018-02-22 04:49:23 (INFO):	Trader Received advice to go long. Buying  ETC
2018-02-22 04:49:25 (INFO):	Poloniex portfolio:
2018-02-22 04:49:25 (INFO):		 BTC: 0.007276840000
2018-02-22 04:49:25 (INFO):		 ETC: 0.000000000000
2018-02-22 04:49:25 (INFO):		 **Buy orders currently restricted to 0.0035 BTC**
2018-02-22 04:49:25 (INFO):	Attempting to BUY 1.05999576001696 ETC at Poloniex price: 0.00330189
2018-02-22 04:54:23 (INFO):	In uptrend since 3 candle(s): @ 0.00330012
2018-02-22 04:59:23 (INFO):	In uptrend since 4 candle(s): @ 0.00330825
2018-02-22 05:04:24 (INFO):	In uptrend since 5 candle(s): @ 0.00330826
2018-02-22 05:11:04 (INFO):	In uptrend since 6 candle(s): @ 0.00330368
2018-02-22 05:15:24 (INFO):	In uptrend since 7 candle(s): @ 0.00330673
2018-02-22 05:19:04 (INFO):	In uptrend since 8 candle(s): @ 0.00330635
2018-02-22 05:19:27 (INFO):	BUY was successful after the first attempt
2018-02-22 05:19:27 (INFO):	**Buy orders currently restricted to 0 BTC**
2018-02-22 05:19:28 (INFO):	(PROFIT REPORT) original simulated balance:	 0.00727684 BTC
2018-02-22 05:19:28 (INFO):	(PROFIT REPORT) current simulated balance:	 0.00727632 BTC
2018-02-22 05:19:28 (INFO):	(PROFIT REPORT) simulated profit:		 -0.00000052 BTC (-0.00714709%)

Just a minor bug with the message Buy orders currently restricted to 0 BTC

advice('buy/sell', price, amount)

@aapsoftware sorry still haven't understood your needs. how would you be able to dynamically specify the amount you want before each transaction.

Just a minor bug with the message Buy orders currently restricted to 0 BTC

That's expected. After you have used up the 0.0035 BTC that you wanted your bot limited to, then the bot should not be able to execute further buy orders. However if a sell order get executed, then any currency that comes of that becomes tradable again. In this way, the other 0.0037 BTC you have in your exchange balance never gets touched.

@Ali1 I would like to have the option to either let the bot trade the full amount available (which is the current behaviour) or specify exactly how much I want to trade - consider this example
- the limit amount is 0.004BTC

advice (buy, 0.002) will  only trade half of the limit amount.
This is the first step towards making it possible to implement more flexible  strategies, where you can:
- place an order at a specific price, before the market reaches that price (i.e. 10% above current market price)
place multiple, consecutive buy/sell orders (currently you can only do buy, sell, buy, sell and so on. I would be nice to have the possibility to do buy, buy, buy, sell, buy, sell, sell , etc.

Right, now I understand you want the strategy to decide how much to buy. I recommend you create an issue or PR and see what discussion comes out of that.

@aapsoftware I know exactly what you're talking about, you want to go in with different position sizes which are defined by the strategy, that would be a very good feature.
This PR is not at all about that, what you're wanting would be a layer built on top of this. What this feature is about is limiting the maximum amount which can be traded by the portfolio manager, so that for instance you could have multiple strategies running at the same time on the account which all had a maximum trade limit measured in the base currency.
BTW, @askmike @Ali1 I'm actively working on this feature now including a separate trade class responsible for getting a trade executed, which could be sub-classed to enable different trade execution strategies, will submit a PR after testing.

I've been testing this feature for about a week and so far everything looks fine. There is one thing I would change though.  I'd split the trader.orderUpdateDelay into an delay for buy and one for sell. If you think about it makes sense - you want to re-enter a buy order fairly quick, while for a sell order you may be willing to hold out longer.

This is up to @askmike ultimately if this is merged, although we are working on reworking the structure of how trades are handles to allow a more robust framework for trades and orders starting with this PR https://github.com/askmike/gekko/pull/1968. Reguardless, to maintain backward compatibility, I'd suggest we setup the parameter names so they can be consistent with the new developments.
To maintain consistency then I suggest to change the currencyAmountAvailableToTrade parameter name to currencyAllowance. Thoughts?

@ansonphong  currencyAllowance is a shorter name and it would make the code easier to read. I think It's a good idea.

@aapsoftware @ansonphong @Ali1 I am not comfortable with the amount of if checks which lead to different codepaths based on configuration and or portfolio situation, without any debug statements in here it becomes extremely hard to figure anything out once something goes wrong: we need to know how Gekko was configured and estimate what choices the code made at the time to come to any sort of conclusion, let alone actually fix it.
I want this feature, but the way this is now implement scares me from a maintenance point of view.

@askmike valid point (the debug statements). The code as is could use some more logging for when things don't go as expected.
About the ifs perhaps it would be better if the data paths would be decided at initialisation; you could have multiple methods and decide which one to use an initialisation, depending on the config options. It would also be a bit more efficient, but the code does require a bit of rework (which is well worth doing) to avoid repetition.
What do you think?

About the ifs perhaps it would be better if the data paths would be decided at initialisation; you could have multiple methods and decide which one to use an initialisation, depending on the config options.

Yes this a 100%: calculate all values in the beginning and log them (in debug or maybe even more). After that the codepath can stay the same for all everyone and there are no situations where the code goes down  a different path.

@askmike Yes having spent several hours studying this PR, I understand where you're coming from regarding maintenance.
I'm currently testing this PR on 4 live gekkos and it's running as expected, it had no merge conflicts with the develop branch, so should be easy to get working for any early adopters who want to use it in the meantime. @Ali1 thanks for putting this together.

I just tried your code, and after few trades it forget the config and start to trade with whole portfolio.
https://pastebin.com/dN34KMRF

Hi @weeder
Looks like the problem happens at 2018-03-08 21:15:38. A buy order is partially filled before new trade advice to sell is received. Gekko forgets to take into account the partially filled order before going on to try and sell. This is because calculations about trade allowance are only made after the order fills completely should be expected to happen. But of course there will be times when it does not so perhaps you or someone could provide a fix for these cases since I am no longer able to be involved at the moment.
It does look like your strategy makes a lot of buy and sell decisions quickly and so really, these orders shouldnt be taking so long to fill. You can look at your configuration and also I think another issue/PR is looking into adding more options to allow orders to get filled faster.

@Ali1 I've been running this PR live, and it makes a successful first buy, though it's unable to finish a second trade if one of the orders wasn't filled, it seems as though the currencyAmountAvailableToTrade is being set to 0 here and never changed back later. Is this the intended behavior?
See plugins/trader/portfolioManager.js line 116

It will be great to add something to this feature like for exemple say max amount by trade is 50 USD.
If a buy occure it buy 50 USD then if the rate down and an other long advice happen it buy again 50 USD ...
I want develop a strategy which buy limited amount but not just one time.
I don't know if it possible ...

I trade buying many orders as long as price go up, this minimize the risk of teh operation. A way to create this kind of settup in geckko will be great!

I trade buying many orders as long as price go up, this minimize the risk of teh operation. A way to create this kind of settup in geckko will be great!

Creating the orders is easy, reporting this in a simple way in the UI is a lot harder. Unforutnately this is out of scope for now.
--
ALL: This PR was perfect to get the ball rolling, I'm working on a massive upgrade for the execution logic that will include all func described in this PR. See #2118. Let's keep all still alive discussion either in there OR on the forum.


Infinite Respone

Type: Bug
I asked Github Copilot to convert a mel script into Python in Maya and its reponse a lot until I asked it to stop
expandPolyGroupSelection;
polyCleanupArgList 4 { "0","2","1","0","0","0","0","0","0","1e-05","0","1e-05","1","1e-07","0","-1","0","0" };
Extension version: 0.21.2
VS Code version: Code 1.94.2 (384ff7382de624fb94dbaf6da11977bba1ecd427, 2024-10-09T16:08:44.566Z)
OS version: Windows_NT x64 10.0.22631
Modes:

System Info

Item
Value

CPUs
Intel(R) Xeon(R) w5-2445 (20 x 3096)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabledwebnn: disabled_off

Load (avg)
undefined

Memory (System)
127.25GB (92.87GB free)

Process Argv
--crash-reporter-id a4649202-1f1b-4c71-9571-ef46a36606ab

Screen Reader
no

VM
0%

Hi, are you guys still working on this issue?

It's an issue we are aware of but it's caused by the LLM which is largely out of our control. Is this something you are hitting often? Do you have further examples we could use for reproducing it?

Hi @lramos15, I don't meet this often and can't reproduce it now.


Implement the blackjack strategy card

Add a feature where the player can take a peek at the strategy card anytime.

clickable button to bring up the card (will be placed on the bottom left of the screen)
click anywhere to minimize it

Strategy card is now implemented to help guide the player.


feat: Refactor

Takes an eloquent approach to db models

[ ] Check for any unused methods
[ ] Fix and add to tests
[ ] Test app

This pr was really just to refactor the models


Verify Compact mode for your adapter

We have detected that your adapter supports the compact mode. Please use the latest js-controller 2.0 and verify that everything works.
Some more information what is important to check can be found at https://github.com/ioBroker/ioBroker.js-controller/issues/512
On questions please answer to the linked issue. Please close this issue after your test and add the version number that you have tested please as a comment.
Thank you for your support.

test ok with 1.30 and 1.4.0

Thank you!