Variables are not re-evaluated after lookahead

Got another one!
It seems variables might not be re-evaluated properly after a lookahead. For example:
EXTERNAL GET_LINE_COUNT()

Line count: {GET_LINE_COUNT()}

Line count: {GET_LINE_COUNT()}

~ temp should_execute_before = GET_LINE_COUNT()

Line count: {should_execute_before}

-> DONE

Given a GET_LINE_COUNT which prints however many lines have been consumed, inkcpp prints:
Line count: 0
Line count: 1
Line count: 1

Where the C# package prints:
Line count: 0
Line count: 1
Line count: 2

(even with the function bound as lookahead-safe). This can cause issues in games which have a state change after a line is printed.
Here's a unit test.. I've replicated the unit test in C# and confirmed that it produces the second output.
Many thanks for your hard work on this package, by the way!

Thanks for providing a test case :+1

The problem was that we only returned one instruction after a look ahead ...
Now we do as Inkle specifies, with double external execution and an observer who gets called multiple times for the same change ^^
You are welcome to check the PR #73


BatchPromptSchedule missing

Hi,
I have redownloaded it several times with comfyui manager and manually both but it keeps on showing this error. I have also reinstalled and checked everything from requirements.txt.
Need Help!
Thanks,

Yes,
E:\ComfyUI_windows_portable>.\python_embeded\python.exe -s ComfyUI\main.py
--windows-standalone-build
Failed to execute startup-script:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\prestartup_script.py
/ source code string cannot contain null bytes
Prestartup times for custom nodes:
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy
0.0 seconds (PRESTARTUP FAILED):
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
Total VRAM 8192 MB, total RAM 65450 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3070 : cudaMallocAsync
VAE dtype: torch.bfloat16
Using pytorch cross attention
Loading: ComfyUI-Impact-Pack (V4.38.2)
Loading: ComfyUI-Impact-Pack (Subpack: V0.3.2)
[WARN] ComfyUI-Impact-Pack: ComfyUI or ComfyUI-Manager is an outdated
version.
Loading: ComfyUI-Manager (V1.5.2)
ComfyUI Revision: 1795 [a4ec54a4] | Released on '2023-12-08'
Comfyroll Custom Nodes: Loaded
[comfyui_controlnet_aux] | INFO -> Using ckpts path:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts
[comfyui_controlnet_aux] | INFO -> Using symlinks: False
[comfyui_controlnet_aux] | INFO -> Using ort providers:
['CUDAExecutionProvider', 'DirectMLExecutionProvider',
'OpenVINOExecutionProvider', 'ROCMExecutionProvider',
'CPUExecutionProvider']
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\dwpose.py:25:
UserWarning: DWPose: Onnxruntime not found or doesn't come with
acceleration providers, switch to OpenCV with CPU device. DWPose might run
very slowly
warnings.warn("DWPose: Onnxruntime not found or doesn't come with
acceleration providers, switch to OpenCV with CPU device. DWPose might run
very slowly")
Traceback (most recent call last):
File "E:\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1800, in
load_custom_node
module_spec.loader.exec_module(module)
File "", line 940, in exec_module
File "", line 241, in
call_with_frames_removed
File
"E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_FizzNodes_init.py",
line 57, in
from .ScheduledNodes import (
File
"E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_FizzNodes\ScheduledNodes.py",
line 4, in
import numexpr
ModuleNotFoundError: No module named 'numexpr'
Cannot import
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_FizzNodes module
for custom nodes: No module named 'numexpr'
[tinyterraNodes] Loaded
Efficiency Nodes: Attempting to add Control Net options to the 'HiRes-Fix
Script' Node (comfyui_controlnet_aux add-on)...Success!
Efficiency Nodes: Attempting to add 'AnimatedDiff Script' Node
(ComfyUI-AnimateDiff-Evolved add-on)...Success!
Efficiency Nodes Warning: Failed to import python package 'simpleeval';
related nodes disabled.
[rgthree] Loaded 18 exciting nodes.
[rgthree] Optimizing ComfyUI recursive execution. If queueing and/or
re-queueing seems broken, change "patch_recursive_execution" to false in
rgthree_config.json
Searge-SDXL v4.3.1 in
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\SeargeSDXL
Traceback (most recent call last):
File "E:\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1800, in
load_custom_node
module_spec.loader.exec_module(module)
File "", line 940, in exec_module
File "", line 241, in
call_with_frames_removed
File
"E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui_init.py",
line 1, in
from .WAS_Node_Suite import NODE_CLASS_MAPPINGS
File
"E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py",
line 36, in
from numba import jit
ModuleNotFoundError: No module named 'numba'
Cannot import
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui
module for custom nodes: No module named 'numba'
Import times for custom nodes:
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\SD-Latent-Upscaler
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-previewlatent
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Frame-Interpolation
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\Derfuu_ComfyUI_ModdedNodes
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy
0.0 seconds (IMPORT FAILED):
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_FizzNodes
0.0 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes
0.0 seconds (IMPORT FAILED):
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui
0.1 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_tinyterraNodes
0.3 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\SeargeSDXL
0.3 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
0.4 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux
0.8 seconds:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack
Starting server
To see the GUI go to: http://<IP_ADDRESS>:8188
FETCH DATA from:
E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json
On Fri, Dec 8, 2023 at 3:58 PM FizzleDorf @.***> wrote:

Can you show me the com.and line error message? I believe this may be
because of a failed requirements install. The installation instructions in
the readme should have you covered or check the closed issues for the same
missing modules.
—
Reply to this email directly, view it on GitHub
https://github.com/FizzleDorf/ComfyUI_FizzNodes/issues/47#issuecomment-1847328852,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BBERNUNAUCDA2QNBW6J5T7LYIMTIVAVCNFSM6AAAAABAMPSGSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBXGMZDQOBVGI
.
You are receiving this because you authored the thread.Message ID:
@.***>


sourcemap not generated

I am trying to generate a sourcemap so that my huge file with all the client code can be mapped back to the individual files. My config is:
  config.ngAnnotate = {
    options: {
      ngAnnotateOptions: {
        sourcemap: true,
        sourceroot: '/client'
      }
    },
    clientApp: {
      src: config.files.clientApp,
      dest: 'client/dist/app.js'
    }
  };

Unfortunately it does not work and no sourcemap indications appear in the produced app.js file
And yet ngAnnotate should support those options....

grunt-ng-annotate 0.9.0 with source map support released (see description in README).
Please report bugs. :)
BTW, the newest grunt-ng-annotate should work on io.js as well as Node.

Unfortunately, this example from above still does not work. I need the possibility to use an inFile and will have a look at @la-yumba's solution.

Unfortunately, this example from above still does not work in 0.9.2.

It's because the example is incorrect. Source map parameters should be put inside the value at the sourcemap property, not side by side. Try sth like:
  config.ngAnnotate = {
    options: {
      ngAnnotateOptions: {
        sourcemap: {
          sourceroot: '/client'
        }
      }
    },
    clientApp: {
      src: config.files.clientApp,
      dest: 'client/dist/app.js'
    }
  };

As for inFile, I should probably write it at https://github.com/mzgol/grunt-ng-annotate/blob/77d8efc8a9fd0c0f7d963701c8301f5f91a13364/tasks/ng-annotate.js#L156 only if it wasn't explicitly defined. PR welcome or I'll get to it within a few weeks. ;)


feat(iroh-net): DNS queries: lookup ipv6 & ipv4 in parallel

Closes #2006

rather than one single query for both A and AAAA records.

Is that even possible? I didn't see anything in trust-dnss options to allow for such a thing.

rather than one single query for both A and AAAA records.

Is that even possible? I didn't see anything in trust-dnss options to allow for such a thing.

I assumed yes.  But a quick look suggests that's completely wrong and this isn't possible.  So yes, we're doing the best thing we can already.  TIL


Creating target for tvOS

Any tips or a sample project for creating a tvOS target?
I've based my tvOS target on the iOS target.  The target name is Socket-tvOS and I've modified the Project Module Name setting to be "Socket".  Nevertheless, my framework project that contains BlueSocket does not compile because it cannot find the "Socket" module ("No such module 'Socket'").  I've done my best to carefully confirm all of the settings in my own tvOS framework project that utilizes BlueSocket are correct and match the settings for the iOS version of my framework.
Any suggestions?  Thanks in advance.

I should note that the tvOS BlueSocket target I created does compile and produce a target.  It is simply that my framework cannot see it even though it is added to it's target.  My guess is that I've done something wrong in ensuring that BlueSocket tvOS target has the correct module name.

I've added a new tvOS target to the latest version, 0.12.77.  Tested here and it seems to be fine.  Re-open the issue if you have an problems.

Works.  Thank you sir.


T4 code generation doesn't fully support interface inheritance

When an interface marked with [Entity] inherits from an interface not marked with the [Entity] attribute, the generated implementation code is not correct.

Hi
When I'm using brightstardb for entity framework, I still have some unclear questions:

It seems brightstardb doesn't support the interface inherited with new keywords that hidden the same property from base interface
InverseProperty always require a corresponding property is defined on the related interface, but entity framework, it's ok to only have the navigation property on one side of the interface
interface inherited from a generic interface with the generic type params specified.

Thanks

Hi,
I'm a bit pressed for time at the moment so these are just quick "off the top of the head" replies :)

No it doesn't support that currently - I'm not sure how possible it is from the pure T4 template. It might be possible to support this from the Roslyn-based code generation though.

The InverseProperty attribute requires you to name the property that you are the inverse of (so therefore the forward property must exist), but you should be able to use the InversePropertyType attribute though. See http://brightstardb.readthedocs.org/en/latest/Entity_Framework/#inverse-property-type-attribute

This is not currently supported at all. It might make for an interesting feature to look at implementing in the future though.

Hi
Thank you for the detaile information.
so for the question 1, in short, the T4 is just a tool for generating the code so if T4 doens't support it, I'm ok with this, I just not sure if I write the generate code manually, does the BrightstarEntityContext and ReflectionMappingProvider support the interface with new keywords?
Thanks

Hi,
I got the problem yesterday
After some inspection, it appears that the problem is at line 831 :
"foreach(var baseInterface in _interfaceCE.Bases.OfType().Where(HasEntityAttribute))"
Your "HasEntityAttribute" doesn't seem to exist (however there is a method with the same name).
I deleted it and now it works.
I don't want to push that because I don't know the purpose of the HasEntityAttribute property or method but I hope it can help you to look into it.

The .Where(HasEntityAttribute) is just c# shorthand for .Where(x=>HasEntityAttribute(x)), so it is calling that method you found.
If you take that check out, then the generator will process all base interfaces which may potentially include interfaces that you don't want processed. I'm not sure if (or how) that would address the OP's question about the use of the new keyword to override a base interface declaration though... but hey, if it works for you I'm not going to stop you ;-)


COMP: Remove shadow and unused vars

Failing tests due to warnings from ITK not related to this work.


Research Mode

Hello everybody.
I'm Giovane Heleno, founder of ISP.Tools (non-profit free online tools for ISPs).
Firstly, congratulations on the fantastic project. It is a great contribution to the community.
I would like a little help, maybe I'm not seeing something.
I'm doing research work and BGPalerter will help me a lot to filter Ripe data.
I set up the environment in research mode exactly as the documentation requires.
I'm using the latest version available, just yesterday I cloned the repository.
Connects normally and receives the Ripe flow. (I using nodemon to facilitate the tests)
[nodemon] starting `babel-node index.js`
Loaded config: /root/BGPalerter/config.yml
BGPalerter, version: 1.33.0 environment: research
Loaded config: /root/BGPalerter/config.yml
Monitoring everything

My config.yml file looks like this:
configVersion: 2
#environment: production
environment: research
maxMessagesPerSecond: 150000
multiProcess: true

connectors:
  - file: connectorRIS
    name: ris
    params:
      carefulSubscription: true
      url: ws://ris-live.ripe.net/v1/ws/
      perMessageDeflate: true
      subscription:
        moreSpecific: true
        type: UPDATE
        host: null
        socketOptions:
          includeRaw: false
monitors:
  - file: monitorHijack
    channel: hijack
    name: basic-hijack-detection
    params:
      thresholdMinPeers: 3
  - file: monitorPath
    channel: path
    name: path-matching
    params:
      thresholdMinPeers: 1
  - file: monitorNewPrefix
    channel: newprefix
    name: prefix-detection
    params:
      thresholdMinPeers: 3
  - file: monitorVisibility
    channel: visibility
    name: withdrawal-detection
    params:
      thresholdMinPeers: 40
      notificationIntervalSeconds: 3600
  - file: monitorAS
    channel: misconfiguration
    name: asn-monitor
    params:
      skipPrefixMatch: false
      thresholdMinPeers: 3
  - file: monitorRPKI
    channel: rpki
    name: rpki-monitor
    params:
      thresholdMinPeers: 3
      checkUncovered: false
      checkDisappearing: false
  - file: monitorROAS
    channel: rpki
    name: rpki-diff
    params:
      enableDiffAlerts: true
      enableExpirationAlerts: true
      enableExpirationCheckTA: true
      enableDeletedCheckTA: true
      enableAdvancedRpkiStats: false
      roaExpirationAlertHours: 2
      checkOnlyASns: true
      toleranceDeletedRoasTA:
        ripe: 20
        apnic: 20
        arin: 20
        lacnic: 20
        afrinic: 50
      toleranceExpiredRoasTA:
        ripe: 20
        apnic: 20
        arin: 20
        lacnic: 20
        afrinic: 50
  - file: monitorPathNeighbors
    channel: hijack
    name: path-neighbors
    params:
      thresholdMinPeers: 3

reports:
  - file: reportFile
    alertDataDirectory: data/
    channels:
      - hijack
      - newprefix
      - visibility
      - path
      - misconfiguration
      - rpki

notificationIntervalSeconds: 86400
alarmOnlyOnce: false

monitoredPrefixesFiles:
  - prefixes/All.yml
#  - prefixes.yml

persistStatus: true
generatePrefixListEveryDays: 0

logging:
  directory: logs
  logRotatePattern: YYYY-MM-DD
  maxRetainedFiles: 10
  maxFileSizeMB: 15
  compressOnRotation: false

rpki:
  vrpProvider: rpkiclient
  preCacheROAs: true
  refreshVrpListMinutes: 15
  markDataAsStaleAfterMinutes: 120

rest:
  host: localhost
  port: 8011

checkForUpdatesAtBoot: true
pidFile: bgpalerter.pid
fadeOffSeconds: 360
checkFadeOffGroupsSeconds: 30

But I get no alert, no log, nothing, just a "Not valid AS number" warning.
2024-02-29T13:36:01+00:00 info: ris connector connected (instance:9ed4be13-24e6-4177-a681-b78af2ae9d6f connection:487de4e3-62c5-40de-9b70-ecd9e7f46690)
2024-02-29T13:36:01+00:00 info: Subscribed to monitored resources
2024-02-29T13:40:23+00:00 error: Error during transform (ConnectorRIS): Not valid AS number
2024-02-29T13:41:47+00:00 error: Error during transform (ConnectorRIS): Not valid AS number
2024-02-29T13:42:09+00:00 error: Error during transform (ConnectorRIS): Not valid AS number
2024-02-29T13:42:23+00:00 error: Error during transform (ConnectorRIS): Not valid AS number
2024-02-29T13:42:34+00:00 error: Error during transform (ConnectorRIS): Not valid AS number
2024-02-29T13:42:49+00:00 error: Error during transform (ConnectorRIS): Not valid AS number
2024-02-29T13:43:42+00:00 error: Error during transform (ConnectorRIS): Not valid AS number
2024-02-29T13:44:09+00:00 error: Error during transform (ConnectorRIS): Not valid AS number

My prefix file is just like this:
---
<IP_ADDRESS>/0:
   description: All IPv4
   ignoreMorespecifics: false
   ignore: false

"::/0":
   description: Monitoring the entire v6 space!
   ignoreMorespecifics: false
   ignore: false

I tried to add some ASNs in the list, but without success either.
Any help would be of great value to us.
Brazilian regards to everyone.

Oh, in case anyone asks or is curious... Yes, I will develop my own Monitor module for my needs.
I just need to make sure everything is working before then.

Hi @giovaneh,
The error is not related to the system not working. The error says that one of the AS you received in input from the streaming is not a valid AS. It would be interesting to see why, but the functioning should not be affected.
The configuration you are operating is not effective because it enables all monitors but doesn't provide the current prefix-as associations. This is not a problem if you are going to develop your monitor, but you have to comment out the currently available one.
However, I just realized that the example that monitors the entire address space is a really bad example of RIS abuse that we are trying to mitigate. I updated the documentation to reflect that.
I hope this helps.


feat: updates to support mfp_executive - STA-576

Includes refactoring some things to the models folder,
adding index.ts files at each level,
changing things to use BaseNode where applicable,
changing the shape of IfElse to diamond and isParentNode nodes to house.

Updated wfm locally to use this.  The changes that had to be made fell into two buckets:

import BlueshellState from the root instead of from nodes
in some unit tests, had to cast a node to  to be able to call or stub onEvent() (which is now protected)


Update Academic Calendar (2023-2024)

Time Tracking: MA-62
This is done when:

[x] AcademicTerm API Academic Calendar JSON updated to reflect Academic and Administrative Calendar 2023–2024
[x] AcademicTerm API testing PASS in QA
[ ] Academic Calendar deployed to PROD
[ ] AcademicTerm API testing PASS in PROD

Notes:

PROD Deployment Target Date: 2023-09-12

Link top "Academic Calendar" goes to 404.


The ITextModel#setValue-function should support ITextSnapshot

For discarding merge changes the 3wm-editor keeps a snapshot around which it then uses to reset the model. Today this is done quite wasteful by reading the whole snapshot into a string which it later passes to setValue. Instead, the setValue-function should accept a snapshot and skip the intermediate string

Nice. Adopted!


[Frontend][PyTorch] Cast to Long gets translated into cast to float32

I have the following pytorch program:
tree_nodes = indexes
feature_nodes = torch.index_select(self.features, 0, tree_nodes).view(-1, self.num_trees)
 feature_values = torch.gather(x, 1, feature_nodes)

thresholds = torch.index_select(self.thresholds, 0, indexes).view(-1, self.num_trees)
lefts = torch.index_select(self.lefts, 0, indexes).view(-1, self.num_trees)
rights = torch.index_select(self.rights, 0, indexes).view(-1, self.num_trees)

indexes = torch.where(torch.ge(feature_values, thresholds), rights, lefts).long()
indexes = indexes + self.nodes_offset
indexes = indexes.view(-1)

when I compile it into TVM, I get the following interesting trace:
%0 = (%v_operator_map.SklearnLGBMRegressor.nodes_offset, %v_operator_map.SklearnLGBMRegressor.nodes_offset, %v_operator_map.SklearnLGBMRegressor.nodes_offset);
  %1 = concatenate(%0);
  %2 = reshape(%1, newshape=[-1]);
  %3 = take(%v_operator_map.SklearnLGBMRegressor.features, %2, axis=0);
  %4 = reshape(%3, newshape=[-1, 3]);
  %5 = gather(%input, %4, axis=1);
  %6 = take(%v_operator_map.SklearnLGBMRegressor.thresholds, %2, axis=0);
  %7 = reshape(%6, newshape=[-1, 3]);
  %8 = greater_equal(%5, %7);
  %9 = take(%v_operator_map.SklearnLGBMRegressor.rights, %2, axis=0);
  %10 = reshape(%9, newshape=[-1, 3]);
  %11 = take(%v_operator_map.SklearnLGBMRegressor.lefts, %2, axis=0);
  %12 = reshape(%11, newshape=[-1, 3]);
  %13 = where(%8, %10, %12);
  %14 = cast(%13, dtype="float32");
  %15 = add(%14, %v_operator_map.SklearnLGBMRegressor.nodes_offset) an internal invariant was violated while typechecking your program [16:15:28] /Users/mainterl/Develop/tvm/src/relay/op/type_relations.cc:107: Check failed: t0->dtype == t1->dtype (float32 vs. int64) :
; ;

Apparently in line 14 the cast into long is translated into a cast into float32.
To reproduce is you can pull this branch, pip install -e .[extra] and run this test file. I can try to generate a minimal running example if it helps.

Thanks, the cast to long was missing, https://github.com/apache/incubator-tvm/pull/6301 should fix this.

Thanks @masahi for the super quick PR! However the problem still persist. I think there is something else going on because now it casts to long and then to float 32 afterwards!
  %0 = (%v_operator_map.SklearnLGBMRegressor.nodes_offset, %v_operator_map.SklearnLGBMRegressor.nodes_offset, %v_operator_map.SklearnLGBMRegressor.nodes_offset);
  %1 = concatenate(%0);
  %2 = reshape(%1, newshape=[-1]);
  %3 = take(%v_operator_map.SklearnLGBMRegressor.features, %2, axis=0);
  %4 = reshape(%3, newshape=[-1, 3]);
  %5 = gather(%input, %4, axis=1);
  %6 = take(%v_operator_map.SklearnLGBMRegressor.thresholds, %2, axis=0);
  %7 = reshape(%6, newshape=[-1, 3]);
  %8 = greater_equal(%5, %7);
  %9 = take(%v_operator_map.SklearnLGBMRegressor.rights, %2, axis=0);
  %10 = reshape(%9, newshape=[-1, 3]);
  %11 = take(%v_operator_map.SklearnLGBMRegressor.lefts, %2, axis=0);
  %12 = reshape(%11, newshape=[-1, 3]);
  %13 = where(%8, %10, %12);
  %14 = cast(%13, dtype="int64");
  %15 = cast(%14, dtype="float32");
  %16 = add(%15, %v_operator_map.SklearnLGBMRegressor.nodes_offset) an internal invariant was violated while typechecking your program [10:41:33] /Users/mainterl/Develop/tvm/src/relay/op/type_relations.cc:107: Check failed: t0->dtype == t1->dtype (float32 vs. int64) :

Perhaps the add only works over floats and is forcing a cast? No idea..

 Is there any way to check which specific tensor create problems?

 Is there any way to check which specific tensor create problems?

Unfortunately no. This is indeed an annoying issue. Even if you are tracing, we get this warning because it seems (?) the weight and bias parameters from torch are not typed. So most likely the warning comes from tensors corresponding to parameters of conv, dense, batch norm etc

Ok I found the solution, but I think I am hitting some bug. So in the __ini__ of the module I have:
self.nodes_offset = torch.nn.Parameter(torch.LongTensor(nodes_offset), requires_grad=False)

and I the middle of the forward method I am doing:
indexes = indexes + self.nodes_offset

(this is what it throwing the 2 casts and related exception).
Interestingly, if I do:
def forward(self, x):
        self.nodes_offset.long()

(meaning forcing a cast at the beginning of the forward method) it works. If instead I add the cast in __init__ I still get the same error. Is this something related with tracing?

%v_operator_map.SklearnLGBMRegressor.nodes_offset

Could %v_operator_map.SklearnLGBMRegressor.nodes_offset be float (or incorrectly cast to float by TVM)? It seems unless both arguments are long, one of them would be cast to float. If both lhs and rhs are long, it should work as expected.

%v_operator_map.SklearnLGBMRegressor.nodes_offset sounds like a parameter to the Relay function. We may be making an assumption that all parameters are float.

Sure! Any suggestion on how to do that? (I am still getting familiar with TVM, to some extent :) )

yes see my updated comment

%v_operator_map.SklearnLGBMRegressor.nodes_offset: Tensor[(1, 3), int64]

yeah the error also says Check failed: t0->dtype == t1->dtype (float32 vs. int64)

This is the full log in case:
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
WARNING:root:Untyped Tensor found, assume it is float32
fn (%v_operator_map.SklearnLGBMRegressor.values: Tensor[(15, 1), float32], %input: Tensor[(3, 2), float32], %v_operator_map.SklearnLGBMRegressor.features: Tensor[(15), int64], %v_operator_map.SklearnLGBMRegressor.nodes_offset: Tensor[(1, 3), int64], %v_operator_map.SklearnLGBMRegressor.thresholds: Tensor[(15), float32], %v_operator_map.SklearnLGBMRegressor.rights: Tensor[(15), int64], %v_operator_map.SklearnLGBMRegressor.lefts: Tensor[(15), int64]) -> Tensor[(3, 1), float32] {
  %0 = cast(%v_operator_map.SklearnLGBMRegressor.nodes_offset, dtype="int64") /* ty=Tensor[(1, 3), int64] */;
  %1 = (%0, %0, %0);
  %2 = concatenate(%1) /* ty=Tensor[(3, 3), int64] */;
  %3 = reshape(%2, newshape=[-1]) /* ty=Tensor[(9), int64] */;
  %4 = take(%v_operator_map.SklearnLGBMRegressor.features, %3, axis=0) /* ty=Tensor[(9), int64] */;
  %5 = reshape(%4, newshape=[-1, 3]) /* ty=Tensor[(3, 3), int64] */;
  %6 = gather(%input, %5, axis=1) /* ty=Tensor[(3, 3), float32] */;
  %7 = take(%v_operator_map.SklearnLGBMRegressor.thresholds, %3, axis=0) /* ty=Tensor[(9), float32] */;
  %8 = reshape(%7, newshape=[-1, 3]) /* ty=Tensor[(3, 3), float32] */;
  %9 = greater_equal(%6, %8) /* ty=Tensor[(3, 3), bool] */;
  %10 = take(%v_operator_map.SklearnLGBMRegressor.rights, %3, axis=0) /* ty=Tensor[(9), int64] */;
  %11 = reshape(%10, newshape=[-1, 3]) /* ty=Tensor[(3, 3), int64] */;
  %12 = take(%v_operator_map.SklearnLGBMRegressor.lefts, %3, axis=0) /* ty=Tensor[(9), int64] */;
  %13 = reshape(%12, newshape=[-1, 3]) /* ty=Tensor[(3, 3), int64] */;
  %14 = where(%9, %11, %13) /* ty=Tensor[(3, 3), int64] */;
  %15 = cast(%14, dtype="int64") /* ty=Tensor[(3, 3), int64] */;
  %16 = add(%15, %0) /* ty=Tensor[(3, 3), int64] */;
  %17 = reshape(%16, newshape=[-1]) /* ty=Tensor[(9), int64] */;
  %18 = take(%v_operator_map.SklearnLGBMRegressor.features, %17, axis=0) /* ty=Tensor[(9), int64] */;
  %19 = reshape(%18, newshape=[-1, 3]) /* ty=Tensor[(3, 3), int64] */;
  %20 = gather(%input, %19, axis=1) /* ty=Tensor[(3, 3), float32] */;
  %21 = take(%v_operator_map.SklearnLGBMRegressor.thresholds, %17, axis=0) /* ty=Tensor[(9), float32] */;
  %22 = reshape(%21, newshape=[-1, 3]) /* ty=Tensor[(3, 3), float32] */;
  %23 = greater_equal(%20, %22) /* ty=Tensor[(3, 3), bool] */;
  %24 = take(%v_operator_map.SklearnLGBMRegressor.rights, %17, axis=0) /* ty=Tensor[(9), int64] */;
  %25 = reshape(%24, newshape=[-1, 3]) /* ty=Tensor[(3, 3), int64] */;
  %26 = take(%v_operator_map.SklearnLGBMRegressor.lefts, %17, axis=0) /* ty=Tensor[(9), int64] */;
  %27 = reshape(%26, newshape=[-1, 3]) /* ty=Tensor[(3, 3), int64] */;
  %28 = where(%23, %25, %27) /* ty=Tensor[(3, 3), int64] */;
  %29 = cast(%28, dtype="int64") /* ty=Tensor[(3, 3), int64] */;
  %30 = add(%29, %0) /* ty=Tensor[(3, 3), int64] */;
  %31 = reshape(%30, newshape=[-1]) /* ty=Tensor[(9), int64] */;
  %32 = take(%v_operator_map.SklearnLGBMRegressor.values, %31, axis=0) /* ty=Tensor[(9, 1), float32] */;
  %33 = reshape(%32, newshape=[-1, 3, 1]) /* ty=Tensor[(3, 3, 1), float32] */;
  %34 = reshape(%33, newshape=[-1, 1, 3]) /* ty=Tensor[(3, 1, 3), float32] */;
  sum(%34, axis=[2]) /* ty=Tensor[(3, 1), float32] */

So even if the tensor is probably typed, if I remove the initial cast to long the program will fail.

This is the log if I remove the explicit cast.
In `main`:
#[version = "0.0.5"]
fn (%v_operator_map.SklearnLGBMRegressor.values: Tensor[(15, 1), float32], %input: Tensor[(3, 2), float32], %v_operator_map.SklearnLGBMRegressor.features: Tensor[(15), int64], %v_operator_map.SklearnLGBMRegressor.nodes_offset: Tensor[(1, 3), int64], %v_operator_map.SklearnLGBMRegressor.thresholds: Tensor[(15), float32], %v_operator_map.SklearnLGBMRegressor.rights: Tensor[(15), int64], %v_operator_map.SklearnLGBMRegressor.lefts: Tensor[(15), int64]) {
  %0 = (%v_operator_map.SklearnLGBMRegressor.nodes_offset, %v_operator_map.SklearnLGBMRegressor.nodes_offset, %v_operator_map.SklearnLGBMRegressor.nodes_offset);
  %1 = concatenate(%0);
  %2 = reshape(%1, newshape=[-1]);
  %3 = take(%v_operator_map.SklearnLGBMRegressor.features, %2, axis=0);
  %4 = reshape(%3, newshape=[-1, 3]);
  %5 = gather(%input, %4, axis=1);
  %6 = take(%v_operator_map.SklearnLGBMRegressor.thresholds, %2, axis=0);
  %7 = reshape(%6, newshape=[-1, 3]);
  %8 = greater_equal(%5, %7);
  %9 = take(%v_operator_map.SklearnLGBMRegressor.rights, %2, axis=0);
  %10 = reshape(%9, newshape=[-1, 3]);
  %11 = take(%v_operator_map.SklearnLGBMRegressor.lefts, %2, axis=0);
  %12 = reshape(%11, newshape=[-1, 3]);
  %13 = where(%8, %10, %12);
  %14 = cast(%13, dtype="int64");
  %15 = cast(%14, dtype="float32");
  %16 = add(%15, %v_operator_map.SklearnLGBMRegressor.nodes_offset) an internal invariant was violated while typechecking your program [15:30:33] /Users/mainterl/Develop/tvm/src/relay/op/type_relations.cc:107: Check failed: t0->dtype == t1->dtype (float32 vs. int64) :
; ;
  %17 = reshape(%16, newshape=[-1]);
  %18 = take(%v_operator_map.SklearnLGBMRegressor.features, %17, axis=0);
  %19 = reshape(%18, newshape=[-1, 3]);
  %20 = gather(%input, %19, axis=1);
  %21 = take(%v_operator_map.SklearnLGBMRegressor.thresholds, %17, axis=0);
  %22 = reshape(%21, newshape=[-1, 3]);
  %23 = greater_equal(%20, %22);
  %24 = take(%v_operator_map.SklearnLGBMRegressor.rights, %17, axis=0);
  %25 = reshape(%24, newshape=[-1, 3]);
  %26 = take(%v_operator_map.SklearnLGBMRegressor.lefts, %17, axis=0);
  %27 = reshape(%26, newshape=[-1, 3]);
  %28 = where(%23, %25, %27) an internal invariant was violated while typechecking your program [15:30:33] /Users/mainterl/Develop/tvm/src/relay/op/tensor/transform.cc:1646: Check failed: condition != nullptr && x != nullptr && y != nullptr:
; ;
  %29 = cast(%28, dtype="int64");
  %30 = cast(%29, dtype="float32");
  %31 = add(%30, %v_operator_map.SklearnLGBMRegressor.nodes_offset);
  %32 = reshape(%31, newshape=[-1]);
  %33 = take(%v_operator_map.SklearnLGBMRegressor.values, %32, axis=0);
  %34 = reshape(%33, newshape=[-1, 3, 1]);
  %35 = reshape(%34, newshape=[-1, 1, 3]);
  sum(%35, axis=[2])
}

Can you come up with a standalone repro script? I couldn't create a simple test that gives the same error.

I can try. I have pushed the code in this Hummingbird's commit. The test file to run is tests\test_lightgbm_converter.py. If you pull the commit and pip install -e . should work (assuming you have pytorch and lightgbm installed). I will also try to have a minimal script for this, but it might be not that easy.

Thanks, I'll try hummingbird. Compiling decision trees to TVM via PyTorch sounds very interesting!

Thanks! Before were manually translating the PyTorch implementation into Relay, but long term we want to use your PyTorch frontend. Any suggestion or help (or contributions) are welcome! In our experiments with the Relay implementation of the models we were getting about 3x performance improvement against PyTorch, we hope to continue to see a similar gain with the frontend!

ok found the problem. The node_offset is indeed giving an warning at
https://github.com/apache/incubator-tvm/blob/939a42b4e976a41e8513b720421d3c3678493715/python/tvm/relay/frontend/pytorch.py#L2146-L2150
Since default_type is float32, from that point TVM thinks node_offset is float32 tensor. That results in unnecessary cast.
This is the same problem I mentioned earlier about conv weight, bias etc. But since they are float32 anyway, the default type we picked for them doesn't do any harm. Your use case is the first one we met where one of parameters to your model is indeed an integer tensor.
Even though the dtype of nodes_offset tensor is int64, when we traverse a TorchScript graph and get the input type of rhs of add node via inputs[1].type().scalarType(), it returns None.  Surprisingly, doing explicit self.nodes_offset.long() in the Torch module  does make scalarType() of node_offset above int64.
I'll look into a bit more on what's going on with typing of parameter tensors at the Torchscript level.

By comparing two Torchscript IRs, with or without  self.nodes_offset.long(), I now understands what's going on.
With the explicit cast, the rhs input of aten::add becomes the output of aten::to(...), which does cast to int64. So when we query the type of rhs, it correctly returns int64.
  %29 : Tensor = prim::GetAttr[name="nodes_offset"](%3)
  %indexes.1 : Long(1:3, 3:1) = aten::to(%29, %23, %22, %22, %21)
  ...
  %indexes.5 : Long(3:3, 3:1) = aten::add(%indexes.4, %indexes.1, %17)

Without the explicit cast, the rhs input of aten::add is the output GetAttr (%29 below). And it seems Torchscript cannot return the right type when asked the type of an output of GetAttr, so it simply returns None in this case.
  %29 : Tensor = prim::GetAttr[name="nodes_offset"](%3)
  ...
  %indexes.5 : Long(3:3, 3:1) = aten::add(%indexes.4, %29, %19)

Interesting. Do you suggest to open an issue against pytorch? It looks to me that there is no clear solution (beside adding the explicit casting) because this is more a design problem with torchscript. Altough should tracing help? Like if we run records through the ops shluld be able to detect that indeed the tensor is of type long.

I don't know if Torch people would consider this as a problem. Anyway I've just opened https://github.com/apache/incubator-tvm/pull/6311 to workaround dtype of GetAttr. With that change, the explicit cast is no longer necessary. The test case I added there should demonstrate the problem you are having.

Fantastic! This is great, thanks. I will test this and let you know but from your test case it looks that it solves our problem. Thanks!

It worked, no more casts are needed and the warnings disappeared.

#6311 is merged


Deep MNIST for Experts Tutorial accuracy is lower than expected (0.9091)

Environment info
Operating System:

$ uname -a
Linux vvinahradski 4.4.0-23-generic #41-Ubuntu SMP Mon May 16 23:04:25 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 16.04

Installed version of CUDA and cuDNN:
(please attach the output of ls -l /path/to/cuda/lib/libcud*):
Not installed
If installed from binary pip package, provide:

Which pip package you installed.
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0rc0-cp35-cp35m-linux_x86_64.whl
The output from python -c "import tensorflow; print(tensorflow.__version__)".
0.9.0rc0

Steps to reproduce

Copy/past https://www.tensorflow.org/versions/r0.9/tutorials/mnist/pros/index.html#deep-mnist-for-experts tutorial
Run code
Observe that accuracy is around 90% instead of expected 99%

What have you tried?

Increased test dataset 10x times. Accuracy is around 94%

Logs or other output that would be helpful
(If logs are large, please upload as attachment).
Source code:
tut.zip
console output:
log.txt

@keveman Can you take a look?
@vrv, @martinwicke If real this could be another release blocker.

If this doesn't happen in 0.8, then yes, it's a blocker.

@girving taking a look.

Ran a quick test:
python --version
Python 3.4.3+

Relevant details:
python -c "import tensorflow; print(tensorflow.__version__)"
0.9.0rc0

Got the following:
step 19800, training accuracy 0.9
step 19900, training accuracy 0.88
test accuracy 0.9071

Changing the learning rate to (1e-3):
step 19800, training accuracy 0.96
step 19900, training accuracy 1
test accuracy 0.9623

Which looks a little better.

0.8.0 and 0.9.0rc0 get the same accuracy (modulo differences due to random initialization). @kingtaurus thanks for checking the learning rate.

@keveman What is that same accuracy?  This tutorial is supposed to get to 99.2% accuracy.  If it's the same model as models/image/mnist/convolutional.py, that's what it should be getting.

The models look identical except Deep MNIST for experts uses 1024
https://www.tensorflow.org/versions/r0.9/tutorials/mnist/pros/index.html#densely-connected-layer
for first FC layer and Vincent's model uses 512
https://github.com/tensorflow/tensorflow/blob/6431560b7ec3565154cb9cdc9c827db78ccfebe7/tensorflow/models/image/mnist/convolutional.py#L174
Also, Deep MNIST uses Adam, whereas convolutional.py uses momentum
with staircase
learning rate decay
https://github.com/tensorflow/tensorflow/blob/6431560b7ec3565154cb9cdc9c827db78ccfebe7/tensorflow/models/image/mnist/convolutional.py#L245
On Fri, Jun 10, 2016 at 3:24 PM, Geoffrey Irving<EMAIL_ADDRESS>wrote:

@keveman https://github.com/keveman What is that same accuracy? This
tutorial is supposed to get to 99.2% accuracy. If it's the same model as
models/image/mnist/convolutional.py, that's what it should be getting.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/2781#issuecomment-225310626,
or mute the thread
https://github.com/notifications/unsubscribe/AABaHCP2OjbIFIlkinllWA4jlh27W5Chks5qKeQvgaJpZM4Iy5CE
.

Its not the same model:
(1) this model uses tf.train.AdamOptimize(1e-4) instead of (tf.train.exponential_decay combined with tf.train.MomentumOptimizer and starts at a learning rate of 0.01)
(2)
y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
tf.reduce_mean(
        -tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))

is used for loss (rather than tf.nn.sparse_softmax_cross_entropy_with_logits) [this should be identical but just a minor point] ;
(3) regularization is being applied in (models/image/mnist/convolutional.py);
(4) convolutional.py does 10 epochs of batch size 64;
tut.py does 18 epochs of batch size 64;
(5) Finally the different size of the Full connected layers;

@vincentvanhoucke Should the mnist for experts tutorial be the same model as models/image/mnist/convolutional.py?
@keveman When you say the accuracy is the same, is it close to 99.2%?

@girving No, both were close to 90%

@keveman We should probably fix the tutorial to be 99% then, unless there is a compelling reason to make it 10x worse than models/image/mnist/convolutional.py.

In my humble opinion, this issue may indicate regression in core nn code.  I believe tutorial was correct at some point.

There is bug in your code. You are only running the learning step once every 100 batches;
        for i in range(20000):
            batch = mnist.train.next_batch(50)
            if i % 100 == 0:
                train_accuracy = accuracy.eval(
                    feed_dict={
                        x:batch[0], y_: batch[1], keep_prob: 1.0})
                print("step %d, training accuracy %g"%(i, train_accuracy))
                train_step.run(
                    feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

Should be:
        for i in range(20000):
            batch = mnist.train.next_batch(50)
            if i % 100 == 0:
                train_accuracy = accuracy.eval(
                    feed_dict={
                        x:batch[0], y_: batch[1], keep_prob: 1.0})
                print("step %d, training accuracy %g"%(i, train_accuracy))
            train_step.run(
                    feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

Good catch @kingtaurus. I can confirm ~99% accuracy with 0.9.

@kingtaurus
Thank you for catching up that indentation mistake.
When I corrected it, I got the same accuracy they said on the tutorial:  0.9922
step 19000, training accuracy 1
step 19100, training accuracy 1
step 19200, training accuracy 1
step 19300, training accuracy 1
step 19400, training accuracy 1
step 19500, training accuracy 1
step 19600, training accuracy 1
step 19700, training accuracy 1
step 19800, training accuracy 1
step 19900, training accuracy 1
test accuracy 0.9922


MCPing implementation

Describe Problem
finish implementing mcping
Suggest Solution
mcping isnt fully implemented especially not with the new refactor
Additional Details
mcping can be completely and fully implemented with the new discordrb feature that supports attaching files to embedded text

since most of the work is already done, this shouldnt be too difficult to finish


Lost text when switching during profile editing

Subject: Lost text when switching during profile editing
Description:
I changed from editing "About" to "Home" and then back
Results:
After switching all text was discarded and I first realized that after filling out both pages with lots of text :'( Would be nice if save button in more prominent and that changing from "About" to "Home" would not delete any unsaved text until edit is finish. Best would be to show a prompt asking if you want to save before any changes are lost.
Backend version: develop-2a85c9e0
Frontend version: develop-2a85c9e0
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:98.0) Gecko/20100101 Firefox/98.0
Page: https://couchers.org/profile/about
User: maks1 (11856)

#2721 there is now a confirmation if input will be lost


Cleanup subscription maps on unsubscribe

Hi again,
Added some clean-up of the subscription maps to avoid them to grow for ever.
I had problem how to handle if max_messages was used in unsub, but we do not
use that so I appreciate if this also could be merged. Maybe you have ideas for max_messages?
Also added a disconnect test for my earlier commit.

Any comments?

Sorry for the late response and thanks for your contribution. Would it be possible for you to add a test for it?

I don't see an obvious way to test the internal content of the map (the commit solves a memory leak in the case we do a lot of sub,unsub on the same subject from different pids). Would it be enough to just make sure that repeated sub, unsub,sub, unsub still works, that is we have not destroyed the use case where we earlier reused the same key in the maps?

OK, merging this. Thank you for your contribution.


Is it .spin/ai_models or .spin/ai-models?

The text here talks about both .spin/ai-models and .spin/ai_models. Which is it?

Using ai_models returns this error in Python:
AssertionError: Error::RuntimeError("Could not read model registry directory '/Users/technosophos/Code/Python/bookmarker/.spin/ai-models': No such file or directory (os error 2)")

It appears to be .spin/ai-models. Testing that seems to work

Yes, looking at the Spin code it's the hyphen:


Show external link and register here fields

Would also be nice if the plugin could show the value for the 'external (ticket) link' and 'register here' fields.

by @0spinboson

I imagine it is the externalParticipationUrl field we are talking about: https://framagit.org/framasoft/mobilizon/-/blob/4.1.0/schema.graphql?ref_type=tags#L2847

yeah ideally the link would either go to the external registration page or just to the event page on the mobilizon instance so people can register/login or subscribe anonymously as a participant.


feat: MLList.ofTaskList

Take a List of Tasks, and produce a monadic lazy list that returns the values as their become available.
(prerequisite for hint)

I've added a simple test.


Do not require authentication in API

Current authentication behavior is to require an API token.

Most importantly, are the replacements for the current bibxml GET APIs requiring auth?
My full expectation was that we would be able to either

replace the current GET calls to http://xml2rfc.ietf.org/public/rfc/bibxml/](https://xml2rfc.tools.ietf.org/public/rfc/bibxml#/whatever with a GET call to the new service, or
possibly even redirect xml2rfc.ietf.org at some point to the new service.

Perhaps we mean two different things by the term "API"? Are there two sets of APIs, some that require auth and some that do not? As long as the GET API replacements do not, then we're cool.
If not, then there is a major fail IMO.

@TonyLHansen we currently have two APIs:

legacy API that works just like today that returns BibXML. This endpoint does not require authentication. e.g. https://dev.bibxml.org/public/rfc/bibxml-rfcsubseries/reference.BCP.0048.xml
new API that is supposed to be extensible to other formats. This endpoint currently requires Datatracker authentication: https://dev.bibxml.org/api/v1/by-docid/?doctype=IETF&docid=BCP 48&format=bibxml

Excellent. Then we're good. From the earlier descriptions, it had sounded like all access had been changed to require auth. Thank you.

Closing now. Thanks @TonyLHansen @strogonoff !


linux_executable

i modified complete code of "20-20-20-Eyesafe-project.py" file such that it can be executed on linux safely with the same functionalities.
Description
i used library name "subprocess" instead of "win32gui" and "wind32con" library

Related Issue

Fixes # (issue)
fixed issue -: script was not supported on linux.
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

How Has This Been Tested?
i tested this particular script on ubuntu and fedora distros and it works completely fine
Checklist:

[x] I have registered myself at Contrihub website.
[x] My code follows the code style of this project.
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] Any dependent changes have been merged and published in downstream modules
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Test Configuration:

Firmware version:
Hardware:
Toolchain:
SDK:

hey @PriyavKaneria i think contrihub website is not opening, i cant be able to submit PR link from my profile there.

#3 fixed


Question: Circles on line chart

Is there any way i can implement https://www.highcharts.com/demo/line-basic
basically, i am looking to have a circle on each end point and also to have a style change on hover.
I can't find any example like this in the documentation. Can you suggest how I can achieve this?
Thanks.

Actually, i can use SactterSeries, but i still can't apply hover on the scatter circles.

You are correct the ScatterSeries does not support the hover feature yet.


Help adding Athena service?

I managed to create a separate JS repository to be able to query Athena (using the v3 of the AWS SDK).
I'm not a javascript developer and I'm having trouble importing this new library into Google Apps Script. Could you guys help me out with any tips or reference materials?
Or maybe help me out incorporating my new functions into this brilliant and life-saving repo?
Here's what I came up with:
const { AthenaClient, StartQueryExecutionCommand, GetQueryExecutionCommand, GetQueryResultsCommand } = require('@aws-sdk/client-athena');

async function runAthenaQuery(athenaClient, database, query, outputLocation) {

  const queryExecutionParams = {
    QueryString: query,
    QueryExecutionContext: {
      Database: database,
    },
    ResultConfiguration: {
      OutputLocation: outputLocation,
    },
  };

  const startQueryExecutionCommand = new StartQueryExecutionCommand(queryExecutionParams);

  try {
    const queryExecutionResponse = await athenaClient.send(startQueryExecutionCommand);
    const queryExecutionId = queryExecutionResponse.QueryExecutionId;
    console.log("Query submitted. Execution ID:", queryExecutionId);
    return queryExecutionId;
  } catch (error) {
    console.error("Error executing Athena query:", error);
    throw error;
  }
}

function getQueryResultsWithPolling(athenaClient, queryId) {
  return new Promise((resolve, reject) => {
    const interval = setInterval(() => {
      const queryExecutionParams = { QueryExecutionId: queryId };
      const getQueryExecutionCommand = new GetQueryExecutionCommand(queryExecutionParams);

      athenaClient.send(getQueryExecutionCommand)
        .then((response) => {
          const status = response.QueryExecution.Status.State;
          console.log(`Query execution status: ${status}`);

          if (status === 'SUCCEEDED') {
            clearInterval(interval);

            const getQueryResultsParams = { QueryExecutionId: queryId };
            const getQueryResultsCommand = new GetQueryResultsCommand(getQueryResultsParams);

            return athenaClient.send(getQueryResultsCommand);
          } else if (status === 'FAILED' || status === 'CANCELLED') {
            clearInterval(interval);
            reject(new Error(`Query execution failed or cancelled. Status: ${status}`));
          }
        })
        .then((resultResponse) => {
          resolve(resultResponse);
        })
        .catch((error) => {
          clearInterval(interval);
          reject(error);
        });
    }, 2000);
  });
}

function executeAthenaQuery(database, query, outputLocation, awsRegion, awsAccessKeyId, awsSecretAccessKey) {
  const awsCredentials = {
    accessKeyId: awsAccessKeyId,
    secretAccessKey: awsSecretAccessKey,
  };

  const athena = new AthenaClient({
    region: awsRegion,
    credentials: awsCredentials,
  });

  return new Promise(async (resolve, reject) => {
    try {
      const queryId = await runAthenaQuery(athena, database, query, outputLocation, awsRegion, awsAccessKeyId, awsSecretAccessKey);
      console.log('Query ID:', queryId);

      const queryResults = await getQueryResultsWithPolling(athena, queryId);
      console.log('Query Results:', queryResults);

      resolve(queryResults);
    } catch (error) {
      console.error('Error:', error);
      reject(error);
    }
  });
}

Thanks a ton

Hi @brunopini, see the README. You can add more services, e.g.
npm run sdk --sdk=ses,s3,ec2,lambda,dynamodb && npm run build. I'm not sure what the service name is for athena, but maybe it's just athena. If you only wanted that service, you could add it that way.
As for the actual commands from the SDK, you would have to test it for that individual service and consult the AWS SDK documentation.

@dxdc thanks for the quick reply and pardon my ignorance here -- as mentioned, I'm not really familiar with javascript.
I've read the docs and explored all of your code, but failed to understand the steps.

Where do I npm run? Locally after cloning your repo?
How do I then import the new code into Google Apps Script as a library?
-- I ask this because I tried bundling my own code and adding it to a GAS project, but keep getting a ReferenceError: executeAthenaQuery is not defined.

As for the commands, the code I shared above is executing succesfully, although using V3. I'm sure I can adapt back to V2.
If it's not asking too much, could you provide more info even if only an overview so I can do my own research?

Updates:
I managed to clone this repo and followed your instructions to access the athena service, but I'm struggling to resolve this error:
><EMAIL_ADDRESS>sdk
> echo "Building SDK for AWS Services: $npm_config_sdk" && node aws-sdk-js/dist-tools/browser-builder.js $npm_config_sdk > src-sdk/aws-sdk.js

"Building SDK for AWS Services: $npm_config_sdk"
(node:13968) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy
(Use `node --trace-warnings ...` to show where the warning was created)
C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\dist-tools\service-collector.js:142
    throw new Error('Incorrectly formatted service names');
    ^

Error: Incorrectly formatted service names
    at ServiceCollector (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\dist-tools\service-collector.js:142:11)
    at module.exports (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\dist-tools\transform.js:32:17)
    at nr (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\module-deps\index.js:301:23)
    at C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\module-deps\index.js:283:21
    at onfile (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\resolve\lib\async.js:155:21)
    at onex (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\resolve\lib\async.js:210:32)
    at C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\resolve\lib\async.js:22:20
    at FSReqCallback.oncomplete (node:fs:209:5)

Node.js v18.16.1

I'm positive it is not a typo on the service name, because running with the default services, I still get the error.
Any idea why?
Thanks again

(not sure if I needed to @ you for notifications to apply)
@dxdc

I don't think your V3 code will port to V2. I don't have plans to revisit V3, when I looked before, it seemed more complicated.
Which command was failing for you at the terminal did you run? You didn't paste what you typed in the terminal. I don't think I've tested it with Node 18 though. Maybe try Node 16? It may also be a path issue since I haven't tested this on Windows.
I built the SDK for you as follows, though not sure if you need all of these services. It adds a little bit of bloat, but it's still pretty small.
npm run sdk --sdk=ses,s3,ec2,lambda,dynamodb,athena && npm run build
See the advanced deployment instructions here: https://github.com/dxdc/aws-sdk-google-apps#create-your-own-library
Just use this version of AwsSdk.js instead of the one in the dist folder.
AwsSdk.js.zip


Add Striveworks to adopter list

This PR will add Striveworks to the list of kserve adopters
Proposed Changes

add Striveworks to adopters list

/assign @yuzisun

thanks @JakeNeyer !!
Do you also want to add the logo to the main page ?

@yuzisun that would be awesome! I'll add that to the PR 🙂

Thanks @JakeNeyer !! Welcome to the family !
/lgtm
/approve


Build failure: emacs on macOS 13.4 (aarch64-darwin)

Trying to build the emacs package results in the build hanging on the step make -C ../admin/grammars all EMACS="../../src/bootstrap-emacs". bootstrap-emacs processes are stuck sitting idle.
› ps aux | grep bootstrap
nan              15404   0.0  0.0 408629104   1616 s004  R+    8:39am   0:00.00 /nix/store/27kz2hi95sy9fs3vlkklfcfpkb3h2l7q-gnugrep-3.11/bin/grep --color=auto bootstrap
56930            12232   0.0  0.0 410067936  63168   ??  S     8:37am   0:01.00 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/wisent/grammar -f wisent-batch-make-parser -o ../../lisp/cedet/srecode/srt-wy.el srecode-template.wy
56930            12193   0.0  0.0 410067936  62736   ??  S     8:37am   0:01.01 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/wisent/grammar -f wisent-batch-make-parser -o ../../lisp/cedet/semantic/wisent/python-wy.el python.wy
56930            12022   0.0  0.0 410515424  64176   ??  S     8:37am   0:01.11 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/wisent/grammar -f wisent-batch-make-parser -o ../../lisp/cedet/semantic/wisent/js-wy.el js.wy
56930            11944   0.0  0.0 410515424  63792   ??  S     8:37am   0:01.14 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/wisent/grammar -f wisent-batch-make-parser -o ../../lisp/cedet/semantic/wisent/javat-wy.el java-tags.wy
56930            11937   0.0  0.0 410515424  63456   ??  S     8:37am   0:01.15 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/wisent/grammar -f wisent-batch-make-parser -o ../../lisp/cedet/semantic/grammar-wy.el grammar.wy
56930            11880   0.0  0.0 410646496  63824   ??  S     8:37am   0:01.18 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/bovine/grammar -f bovine-batch-make-parser -o ../../lisp/cedet/semantic/bovine/scm-by.el scheme.by
56930            11718   0.0  0.0 410253280  63952   ??  S     8:37am   0:01.22 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/bovine/grammar -f bovine-batch-make-parser -o ../../lisp/cedet/semantic/bovine/make-by.el make.by
56930            11630   0.0  0.0 410777568  63520   ??  S     8:37am   0:01.19 ../../src/bootstrap-emacs -batch --no-site-file --no-site-lisp --eval (setq load-prefer-newer t) -l semantic/bovine/grammar -f bovine-batch-make-parser -o ../../lisp/cedet/semantic/bovine/c-by.el c.by
56930            11628   0.0  0.0 408859616   4928   ??  S     8:37am   0:00.02 make -C ../admin/grammars all EMACS=../../src/bootstrap-emacs
56930            11355   0.0  0.0 408877024   5664   ??  S     8:37am   0:00.02 make -C ../lisp autoloads EMACS=../src/bootstrap-emacs

Steps To Reproduce
Steps to reproduce the behavior:

Build emacs forcing a rebuild to avoid the cache nix build github:nixos/nixpkgs#emacs --rebuild --print-build-logs

Build log
https://gist.github.com/nanzhong/3bbfda6d7889c5cc68bf92cc81876f4e#file-build_emacs-log
Additional context
A process sample of bootstrap-emacs processes shows that they seem to be stuck after trying to make a call to ns_can_use_native_image_api. I am not sure why that is being called while making grammar parsers...
Link to a process sample: https://gist.github.com/nanzhong/3bbfda6d7889c5cc68bf92cc81876f4e#file-sample_bootstrap-emacs-txt
Notify maintainers
@AndersonTorres @Atemu @jwiegley @lovek323 @matthewbauer
Metadata
› nix-shell -p nix-info --run "nix-info -m"
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
 - system: `"aarch64-darwin"`
 - host os: `Darwin 22.5.0, macOS 13.4`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.16.1`
 - nixpkgs: `/nix/store/xfnvqy8cnn9i4pc1nipw76a41j59ibld-source`

Given that the build farm's macs are able to build it (it's in the cache), this is an issue with your machine.
I'd recommend you to ask on the Discourse whether someone has seen something like this before.
Do you have any antivirus snakeoil running? That might be blocking things.

I've been able to reproduce this on two different machines (one intel Mac which does have antivirus and other endpoint device management software, and this arm Mac with no antivirus or anything else) both with fresh installs of nix. I did install nix on both machines with https://github.com/DeterminateSystems/nix-installer (v0.9.1), I'm not familiar with installation differences between the official installer and that one (aside from the documented differences, which all seem harmless).
One thing of note is that this does not happen when I build emacs-nox. In particular it's the withNS option that causes this.
Thanks for the discord suggestion, I will reach out.

Perhaps try the regular Nix installer.
*Discourse. Though there is an unofficial Discord too I believe. I don't like Discord very much though.
I just tried to reproduce the error but couldn't. eaebbdf59ac1c9dd72011854d8bbad18b22e4cff builds emacs fine on my aarch64-darwin machine on 13.4.

I had some more time to debug this and I've discovered that the reason for this failure is actually auto-allocate-uids. I'm not familiar enough with nix or that experimental feature to have a sense for why the would cause this type of build issue, but leaving it here for posterity if anyone else runs into this issue.


Staging ECS tasks are failing to load

Description
Since at least April 1st our ECS tasks for the staging cluster have been unable to load during the allowed startup period, and have been repeatedly failing health checks and cycling out.
Based on ECS task logs it appears that our health_check_grace_period_seconds terraform variable is not being respected and/or we are experiencing out-of-memory issues
Logs
We noticed a few errors like the following in the ECS task logs for the cycling tasks as well (not sure if this is related or irrelevant):
[Nest] 28 - 04/12/2021, 8:57:59 PM [TopologyService] Unable to download static files from for s3://global-districtbuilder-dev-us-east-1/regions/US/WA/2020-10-22T14:34:20.488Z/: socket hang up +12622ms

After manually updating the health check timeouts associated with the staging load balancer to the maximum allowed, the ECS tasks began failing with an out-of-memory failure instead:
2021-04-13 12:25:35
<--- Last few GCs --->
2021-04-13 12:25:35
[29:0x49abe30] 721721 ms: Scavenge 17699.9 (17802.7) -> 17697.9 (17816.2) MB, 74.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
[29:0x49abe30] 721945 ms: Scavenge 17713.9 (17816.2) -> 17715.8 (17818.2) MB, 75.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
FATAL ERROR: Scavenger: semi-space copy Allocation failed - JavaScript heap out of memory
2021-04-13 12:25:35
<--- Last few GCs --->
2021-04-13 12:25:35
[29:0x49abe30] 721721 ms: Scavenge 17699.9 (17802.7) -> 17697.9 (17816.2) MB, 74.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
[29:0x49abe30] 721945 ms: Scavenge 17713.9 (17816.2) -> 17715.8 (17818.2) MB, 75.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
[29:0x49abe30] 722700 ms: Scavenge 17715.9 (17818.2) -> 17715.1 (17823.2) MB, 754.4 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
<--- JS stacktrace --->
2021-04-13 12:25:35
Cannot get stack trace in GC.
2021-04-13 12:25:35
<--- Last few GCs --->
2021-04-13 12:25:35
[29:0x49abe30] 721721 ms: Scavenge 17699.9 (17802.7) -> 17697.9 (17816.2) MB, 74.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
[29:0x49abe30] 721945 ms: Scavenge 17713.9 (17816.2) -> 17715.8 (17818.2) MB, 75.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
[29:0x49abe30] 722700 ms: Scavenge 17715.9 (17818.2) -> 17715.1 (17823.2) MB, 754.4 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
FATAL ERROR: Scavenger: semi-space copy Allocation failed - JavaScript heap out of memory
2021-04-13 12:25:35
<--- JS stacktrace --->
2021-04-13 12:25:35
Cannot get stack trace in GC.
2021-04-13 12:25:35
<--- Last few GCs --->
2021-04-13 12:25:35
[29:0x49abe30] 721721 ms: Scavenge 17699.9 (17802.7) -> 17697.9 (17816.2) MB, 74.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
[29:0x49abe30] 721945 ms: Scavenge 17713.9 (17816.2) -> 17715.8 (17818.2) MB, 75.3 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
[29:0x49abe30] 722700 ms: Scavenge 17715.9 (17818.2) -> 17715.1 (17823.2) MB, 754.4 / 0.0 ms (average mu = 0.673, current mu = 0.738) allocation failure
2021-04-13 12:25:35
FATAL ERROR: Scavenger: semi-space copy Allocation failed - JavaScript heap out of memory
2021-04-13 12:25:35
<--- JS stacktrace --->
2021-04-13 12:25:35
Cannot get stack trace in GC.

Closed by #690


Scanning for Secrets in Envs

The tool should be able to find a Secret in Envs.
For Example, for images that are produced from such Dockerfile:
FROM docker.io/library/python:3.8
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

WORKDIR /app
COPY requirements.txt /app/
RUN pip install -r requirements.txt

ENV POSTGRES_HOST=database
ENV POSTGRES_USER=postgres
ENV POSTGRES_PASSWORD=postgres
ENV POSTGRES_DB=shopping_list

COPY . /app/

EXPOSE 8000
CMD ["gunicorn", "--bind", "<IP_ADDRESS>:8000", "app:app"]

It should report the Postgres password.

Hi @53845714nF thanks for raising the issue. This seems like an valid feature.


[QUESTION] Cannot download dependent - 'boringssl'

When I first command cmake ..
Occur Error:
unable to access 'https://boringssl.googlesource.com/boringssl/': Failed to connect to boringssl.googlesource.com port 443:
Operation timed out.
I can't download even if I set the agent.
Which file can I modify to replace the download address of boringssh from https://boringssl.googlesource.com/boringssl/ to https://github.com/google/boringssl

Solved

@lidong-jimi ,
Can you include more details?

What command are you running?
What is the platform being used?

If it is Windows, re-running CMake seems to have solved the problem in the past. OpenSSL building is flaky on WIndows. For Linux/MacOS, ensure the git checkout operation is able to reach the URL.


Auto-generate code docs

Related

https://github.com/meltano/handbook/issues/312
(GitLab conversation about docstring standards): https://gitlab.com/meltano/sdk/-/issues/146

Relates to #5949

For this we would want to use either sphinx-autodoc if we are using Sphinx and reStructuredText, or mkdocstrings if we are using Mkdocs and Markdown. I recommend we use mkdocstrings.

For this we would want to use either sphinx-autodoc if we are using Sphinx and reStructuredText, or mkdocstrings if we are using Mkdocs and Markdown. I recommend we use mkdocstrings.

@WillDaSilva note that we can make markdown work in Sphinx with Myst-Parser, and we use that in the SDK: https://github.com/meltano/sdk/blob/main/docs/implementation/index.md?plain=1#L5-L16


Add to documentation: change default port of webhook when using hostNetwork and default Kubelet port setting

As requested in the answer to my issue to the cert-manager repository, I am filing an issue in the website repository https://github.com/jetstack/cert-manager/issues/3472
problem
I am using a kubernetes setup based on my own bare-metal servers where:
pod networking is provided by callico-flannel (canal)
apiserver communicates to (kublet) nodes over using host network
apiserver is not attached to pod overay network and therefore cannot cummunicate with pod over pod network
In order for the apiserver to communicate with the cert-manager webhook, the webhook pod needs to run on the Hostnetwork. (hostNetwork: true)
this is documented on https://cert-manager.io/docs/installation/compatibility/
The default port of 10250 however confilcts on our setup with the default port of the kubelet. If i understand correctly port 10250 is the default port used by the api server to communicate with the kubelet. The default hostnetwork port for cert-maanger webhook of 10250 is therefore alreay in use. This can be seen in the log of the webhook pod, that shows the 'port already in use' error.
https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/
The solution for us was simple. We just changed the securePort for the cert-manager webhoook to 10260.
To be honest, i am a little bit confused why port 10250 was chosen as a default port. As far as I can see this would conflict in any invornment if the kubelet is installed with default port settings. But could be wrong on this.
Describe the solution you'd like
Add to the documentation of using the webhook in combination with the hostnetwork option, that this might require to change the default port of the webhook to a different port. Because the default port of 10250 might conflict with the default port of the kubelet.
helm install cert-manager \
--namespace cert-manager \
--version v1.0.4 \
--set webhook.hostNetwork=true,webhook.securePort=10260 \
jetstack/cert-manager

@niektemme If you have time to contribute a PR with your proposed documentation improvements, I'll help review and merge it. Thanks.
/kind documentation

@niektemme If you have time to contribute a PR with your proposed documentation improvements, I'll help review and merge it. Thanks.
/kind documentation

Thanks for raising this issue! (I realise it was quite a while ago :grin: )
This is actually documented on our site already, although it could be more discoverable: https://cert-manager.io/docs/installation/compatibility/
We can't really change the default port for backwards compatibility reasons - if we change the port and someone's using a port allowlist, we could get blocked.
Gonna close this issue since this is documented! Thanks again!


when exporting a study, files having the same original name will overrite each other

not realistic scenario that the uploaded files have the same name.


Don't render undefined element links

Some elements are not defined but still rendered (probably cause of bad fixtures ?).
First step is to avoid rendering of non defined elements.

done, thanks for review @maximebiloe !


Incorrect type inference for generic types with 'prototype' property

Bug Report
🔎 Search Terms

generic type prototype inference

🕗 Version & Regression Information

This changed between versions 4.7.4 and 5.2.0
This is the behavior in every version I tried, and I reviewed the FAQ for entries about prototype

⏯ Playground Link
Playground link with relevant code
💻 Code

interface Type<T> {
    new(...args: any[]): T;
}
interface Proto<T> {
    prototype: T;
}

function injectWithType<T>(a: Type<T>): T { return null as T };
{
    const okUnknown = injectWithType(Set); // Set<unknown> ⚠
    const okNumber = injectWithType<Set<number>>(Set); // Set<number> ✅
    const okNumber2 = injectWithType(Set<number>); // Set<number> ✅
}

function injectWithProto<T>(a: Proto<T>): T { return null as T };
{
    const okAny = injectWithProto(Set); // Set<any> ⚠
    const okNumber = injectWithProto<Set<number>>(Set); // Set<number> ✅
    const unexpectedAny = injectWithProto(Set<number>); // Set<any> ‼
}

The compiled version has a bug too - double parentheses (reproduced both with the injectWithType and the injectWithProto.):
const unexpectedAny = injectWithProto((Set)) // >=<EMAIL_ADDRESS>const unexpectedAny = injectWithProto(Set()) // <<EMAIL_ADDRESS>
🙁 Actual behavior
When using an interface or  atype with the prototype property to declare class, generic type is getting wrong, especially in the ExpressionWithTypeArguments (‼) case.
🙂 Expected behavior
(⚠) - These lines should have the same type
(‼) - This line should correctly infer generic type
const arrOfNum = injectWithProto(Set<number>) // should be Set<number>, not Set<any>

There's a lot going on here, could you reduce to a single bug report?
The stuff about the emitted JavaScript seems like a separate topic entirely... and I don't see how that's supposed to be a bug either; injectWithProto((Set)) is weird but does it do something incorrectly somewhere?  And since instantiation expressions (#47607) were introduced in TS4.7, I wouldn't have any expectations at all for what JS is emitted if you try to use them on older versions.  Maybe you could remove that part and be explicit about what specific single bug you're reporting here?
Are you saying that the type of Set.prototype should be Set<unknown> instead of Set<any>? Are you saying that the type of (Set<number>).prototype should be Set<number> instead of Set<any>?  Which one of those is the bug you're reporting?

@jcalz I'm reporting about type issue mainly. Both result types below should be equal Set<number>, not only the first one:
injectWithProto<Set<number>>(Set)
injectWithProto(Set<number>);

The compiled version is not breaking, but it's weird and might be related to type inference issue I'm repporting.

It's unrelated, so having it here is distracting at best.  It also seems like you don't care primarily about Set<unknown> vs Set<any>, so maybe that's also just distracting.
If I were filing this issue it would be as a suggestion (not a bug report) that (Set<number>).prototype be of type Set<number>, or more generally that using an instantiation expression on a class constructor would make the prototype property have the same type as its instance type.
The typing of prototype is weird in TS anyway; we pretend that a class prototype has the same type as an instance, even though this is very likely untrue for any class with instance properties.  And if you have a generic class Foo<T> then it has a single prototype Foo.prototype which would therefore need to be the the prototype for all possible instantiations of T, like an infinite intersection Foo<A> & Foo<B> & Foo<C> & ⋯, or forall T. Foo<T>... and Foo<any> is really the closest thing we have to that.  But since class Bar extends Foo<X> {} has Bar.prototype of type Bar, it's not unreasonable to expect that (Foo<X>).prototype would be of type X also.  I'd still be phrasing this as a feature request though.

@jcalz The main issue is that I expect the same behavior for both function calls injectWithProto, but they are completely different. I care only about this.

...Both result types below should be equal Set<number>, not only the first one:
injectWithProto<Set<number>>(Set)
injectWithProto(Set<number>);

I'm confused too. Different lines of code may well have different behavior; that's how stuff gets done. Which line is wrong, and why? Can you post a straightforward example with no other things going on?

@jcalz oh, I understood... SetConstructor.prototype is Set<any>, maybe it really should be a suggestion as you wrote.
@RyanCavanaugh, the real world example how I catched this bug, but simplified. There are 2 files.

This isn't an actionable description of a bug.


Error code when trying to run bot

Whenever I try and open my bot, I get this error message:

The same thing happens with my friends bots. Only a few hours ago, they were working just fine.

Update.

Please download the latest release and copy everything but the data folder into your current bot folder.

Huh, I wonder why it would be telling me to update now instead of earlier. Thanks for the help though!

@Kwoth please close this.

I hate to thread hijack, but @xnaas can you explain what you mean by "copy everything but the data folder into your current bot folder"? Because I updated, but I'm still getting the error, and I don't think I understand what I'm supposed to be copying (or where it's located).


Move notes as desired (position & column)

When the notes are moved, it would be cool if we could change the position as well as the stacking. So just move the notes between other notes or columns.

*also to change the position in the stack. For example from child note to parent note.


Update change password API

Modify the changing password API to support that admin user can change the password of normal users without old password

Fix #840 and #1992.

Coverage decreased (-0.01%) to 64.531% when pulling 76274dbf8459344621c367360b109672de69b756 on ywk253100:180522_change_password into 9989a67d092fa64347dae5217b984a1a68c7ab90 on vmware:master.


Add initial support for transforming data through databases

Currently, the data transformation tools are limited to extracting tar archives, which is useful, but there are a number of tasks that would be helped by the ability to load data in to a database, and then dump back out, all managed through an programmatic api.
Possible tasks that this functionality could be used for:

Transforming the plain data extracts for PostgreSQL in to the custom format, which would hopefully support faster and more configurable restores
Modifying the data within the extract, for example, to remove sensitive data

This has kind of been done for the data-snapshots feature :tada: neither of the possible tasks mentioned have been done yet though...


Pricing Page

@anushbhatia  I would love to work on this issue. Please assign me this issue.

@anushbhatia Can I have the product details and the pricing details, please.

@anushbhatia Can you please provide me the product details and pricing details on LinkedIn, or should I add dummy text for now?

Add Dummy values for now. For reference have a look at any small scale AI startup's pricing if you wish


RainofSnow: Add filters and Fix dates

Adds basic filters.
Fixes invalid date issue, closes https://github.com/LNReader/lnreader-plugins/issues/1154

Still no chapter

Pressing the 1 doesn't work?

Oh i forgor this is pagedplugin xD

I moved logic for page 1


Update es-419.coffee

I´ve translated to spanish more content. I´m sure it is ok on Latam context.

Thank you, @diegotoala :+1:


Fix jobs logs expiration interference with cluster coordination

Summary of the changes / Why this improves CrateDB
If the jobs_log_expiration setting was set the cluster discovery
didn't work as the scheduled task which cleans up entries from the jobs
log conflicted with the cluster coordination tasks as they were using
the same scheduler.
This adds a dedicated scheduler to the JobsLogService to avoid any
interference.
Checklist

[x] User relevant changes are recorded in CHANGES.txt
[x] Touched code is covered by tests
[x] Documentation has been updated if necessary
[x] CLA is signed
[x] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in CHANGES.txt
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

If I increase the delay in
https://github.com/crate/crate/blob/21d4869454dc9a7bb8e527770d585459b35d1049/sql/src/main/java/io/crate/execution/engine/collect/stats/JobsLogService.java#L275
it also works with the ThreadPool scheduler. The scheduler does some ordering
(It internally uses a queue, which probably orders on the ScheduledFuture
(which implements Delayed which in turn implements Comparable<Delayed>)
So I suspect there is a race where the jobs eviction task would always get
prioritized higher (and I think the scheduled coordination even has a back-off
policy, so it gets less priority over time - but didn't double check that)
I'm wondering if we should look into other ThreadPool scheduler usages as well
and migrate them away as well to not mess with cluster coordination tasks.

I think revising what we schedule in the pool used by the cluster coordination is something we should do asap.


Add Issue and PR Template

Inspired from https://github.com/thepracticaldev/dev.to Issue and PR template
/cc @bdougie

This is great. Happy to merge this and evolve it as we see fit.


Env open with Flags::MdbNoSubDir flag requires target file to exists

I'm trying to create a DB like "some_dir/custom.mdb". Where "custom.mdb" is a file, not a directory. I use Flags::MdbNoSubDir to achieve it.
The current implementation requires the target file to exist due to the use of canonicalize, even though underlying mdb_env_open works without creating the file.
It works if I create an empty file manually, but it would be great to not touch the DB file explicitly.

I had the same issue, as a new Rust user the error message was super confusing Error: Io(Os { code: 2, kind: NotFound, message: "No such file or directory" }).

Thank you @shedar and @missinglink for the feedback.
Unfortunately, heed tries to make LMDB safe to use, and one limitation of LMDB is that one process must not open the same environment twice. Therefore, heed ensures that the targeted path doesn't point to the same env by canonicalizing it.
I can propose different solutions to fix this issue:

Keep the current behavior but document it on the Flags::MdbNoSubDir. This forces you to create the file. I could also improve the error returned.
Introduce a new EnvOpenOptions::no_sub_directory that will make the open to create the file for you. I'm not too fond of this solution as it is unsafe, according to the LMDB documentation.

Do not have open an LMDB database twice in the same process at the same time. Not even from a plain open() call - close()ing it breaks flock() advisory locking.

Introduce an unsafe EnvOpenOptions::no_canonicalization or no_uniqueness_check that will make the open method not check if the path is already open but with another path e.g. using symlinks.

What do you think about those solutions? Do you have more solutions to propose?

@Kerollmops
I was thinking about something like this:

Try to canonicalize the path as it is now;
If there is a "No such file or directory" error and self.flags contains Flags::MdbNoSubDir  => Try to canonicalize the path without the last component (without file name)
If there’s still an error - return it
if Flags::MdbNoSubDir in flags and the second canonicalize is successful (parent directory exists and canonicalized) => try to open the “{canonicalized_dir}/{orignial_file_name}” after acquiring write lock

This way, the lock prevents parallel attempts to create the file. At the same time, we only do additional file operations within the lock when opening non-existing env, so it shouldn’t be a performance hit for regular flow. And it doesn't require additional options.
The edge case I see is the following scenario:

heed checks - file doesn’t exist
another process creates a symlink with the same file name before heed opens the file
later, the app opens another env using a target file of the symlink created by a different process above.

But I have a hard time coming up with a real-world scenario where this edge-case flow makes sense.
What do you think?

Your solution is perfect for me. I implemented it in https://github.com/meilisearch/heed/pull/158, but only the great lines. Can you review it and test it? Be careful. It is on the main branch, part of the v0.20.0 version. A lot of breaking changes are coming for the best.

@Kerollmops Thank you for the quick implementation. I've tried the PR branch with my test project - works great.
Also, updating to v0.20.0 was easier than I expected.
BTW, do you have any plans for #41 any time soon? I see it's not in the v0.20.0 milestone for now.

BTW, do you have any plans for #41 any time soon? I see it's not in the v0.20.0 milestone for now.

The amount of work to implement it in a correct and idiomatic way would be too much, but could you propose something on #41 and help me? I also see an issue with the Poly/Database::get method. Should it be changed to return an Iterator, and therefore, should we introduce a new type?

Hey @missinglink,
Thank you very much for this feedback. So what you recommend could be something like this: If the NoSubDir option is set, pop the file name, canonicalize the path and store it with the file name. If the NoSubDir option is missing, then do the same but consider data.mdb as the file name.
This way, we will only canonicalize the path once and always store the environment under the data file itself. We can only open the same environment once with different options.
What do you think about this?

That sounds reasonable, I'm just unclear about the naming convention 'data.md'.
I wonder where this is defined and whether the extension is influenced by the user path 🤔
I can have a look through the C source tomorrow and post links.

That sounds reasonable, I'm just unclear about the naming convention 'data.mdb'.
I wonder where this is defined and whether the extension is influenced by the user path 🤔

It looks like the names are static and will always be the same when the NoSubDir option is not specified.

Yeah nice, FWIW checking the paths isn't ever going to be perfect, what you'd really need to do is check that the two paths refer to the same device and inode as per https://github.com/BurntSushi/same-file
It's a convoluted example I admit, but you can open a database as A.db move it in the terminal using mv to B.db and then open it again at the path B.db, which is something the inode comparison would pick up.
This is one of those things that's actually way trickier than it initially seems.


No way to search within a note

I can't find the way to search for text within an open note. Ctrl+F does not work. For the web version of Evernote this funcionality is provided by browser search.

Hey,
Good feature request. Thanks!

i would also love to have this feature.


Library - relabel 'Class' as 'Call Numbers'

Please reclassify the  field as 'Call Number'
e.g. Record ID B19436 currently has "Additional Identifiers: 7 STIR(J).C" but we'd prefer "Class: 7 STIR(J).C":
e.g. https://linked-art.library.yale.edu/node/378d0828-6b60-497b-8c02-97ddace47f9d
"identified_by": [
{
"type": "Identifier",
"content": "NJ18.St51 J33 2010",
"classified_as": [
{
"id": "http://vocab.getty.edu/aat/300311706",
"type": "Type",
"_label": "Call Number"
}
]
},

I found "7 STIR(J).C" on /library/work/0300177283, it comes from <class>7 STIR(J).C</class>
The default for  that is not processed by any special rules is for it to become an identifier, but there's nothing to inform what a classified_as on that identifier would be, so it ends up just like:
    {
      "type": "Identifier",
      "content": "7 STIR(J).C"
    }

(see this alongside other IDs below)
What information can I use from the record to know to classify this as Call Number?
{
  "@context": "https://linked.art/ns/v1/linked-art.json",
  "id": "https://data.paul-mellon-centre.ac.uk/library/work/0300177283",
  "type": "LinguisticObject",
  "_label": "Stirling and Gowan : architecture from austerity to affluence",
  "language": [
    {
      "id": "http://vocab.getty.edu/aat/300388277",
      "type": "Language",
      "_label": "en"
    }
  ],
  "classified_as": [
    {
      "id": "http://vocab.getty.edu/aat/300263751",
      "type": "Type",
      "_label": "Text"
    }
  ],
  "identified_by": [
    {
      "type": "Identifier",
      "content": "0300177283",
      "classified_as": [
        {
          "id": "http://vocab.getty.edu/aat/300435704",
          "type": "Type",
          "_label": "System-Assigned Number"
        }
      ]
    },
    {
      "type": "Name",
      "content": "Stirling and Gowan : architecture from austerity to affluence",
      "classified_as": [
        {
          "id": "http://vocab.getty.edu/aat/300404670",
          "type": "Type",
          "_label": "Primary Name"
        }
      ]
    },
    {
      "type": "Identifier",
      "content": "7 STIR(J).C"
    }
  ],

...

Noted that these are not item specific, so aren't really call numbers -- there can be multiple accession numbers with a single class. Adding "Class" from aat


Filter out series with all NaN

CATEGORY
Choose one

[ ] Bug Fix
[x] Enhancement (new features, refinement)
[ ] Refactor
[ ] Add tests
[ ] Build / Development Environment
[ ] Documentation

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

REVIEWERS

Codecov Report

Merging #7332 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master    #7332   +/-   ##
=======================================
  Coverage   64.89%   64.89%
=======================================
  Files         428      428
  Lines       20934    20934
  Branches     2330     2330
=======================================
  Hits        13585    13585
  Misses       7225     7225
  Partials      124      124

Impacted Files
Coverage Δ

superset/viz.py
71.53% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f28027...784ea20. Read the comment docs.


feat(menu): increase nested menu elevation based on depth

Increases the sub-menu elevation, based on its depth. Spec for reference

Please rebase

Rebased @andrewseguin.


How do I create Car by sequence number ?

Thanks for your contributing! Excuse me, I have a question.
In this example, how do I call the createCar() function by sequence number ?
createCar chaincode function - requires 5 args, ex: args: ['CAR12', 'Honda', 'Accord', 'Black', 'Tom'],

It should give the number manually now.

We usually extend the objects we save with a 'docType' property. This property defines the category or type of the objects. You can then query fabric for all objects with this 'docType' and count them. When creating a new car, use this number to build your key.
In the method where you would call stubHelper.putState(), add the docType property to the car object and give it the value: 'CAR'
You could then use the getQueryResultAsList method to find all the objects of that docType:
   const cars = await stubHelper.getQueryResultAsList(             {selector:{ docType: 'CAR'}}    );
And store the car with the sequenced key:
const car = await stubHelper.putState('CAR'+cars.length , verifiedArgs.carsObject);


Inconsistency in showing the Model driven apps with latest COE tool kit

Everytime the Admin | Sync Template v2 (Model Driven Apps)  flow from COE core component solution  run show some apps(but not all) and sometime show no MDA apps.  Below are the screen shots of the outputs from 2 different runs.

Kindly let me know how to get fix this. Moreover we have 50+ envs and not sure how to check which flow had ran for which env out of those 50+ flow runs, unless  we go and check each and every flow.
COE Coresolution component: CenterofExcellenceCoreComponents_1_70_6_managed

Regarding how to know which of the flows is for this envt, you cannot know without opening each run.
But you can trigger the flow again for that envt by editing that environment in CoE.
If you dont know how to edit rows in the COE, this will work for you:

Go in to the Environment entity in the Core solution

Got to Data > Select Environment of Choice > Edit > Edit Maker Name (you'll want to change back later) > Save

Go to Model Driven Apps flow and see it running (may take a few minutes to trigger)

Can you please do this to trigger your envt of interest? Then let's see what it collects

Thanks for update Jenefer. Followed the steps as you mentioned, interestingly this time it showed some more MD apps but not complete list of all apps for that environment.

Can you please:

Validate that the flows are finished running (both the Model Driven App and the Apps):

Then open the data in Excel to see if you have the correct data. From the Powerapps App entity, click Export Data

When you get that Excel file, filter to the environment of interest and see if all the data is there. I always make it a table fwiw

I'm trying to determine if the dashboard or the flow is what is causing the error.

Thanks for your quick response Jenefer. Inconsistent apps data is definitely from flow only, as I am observing in  power platform admin app, sometimes the apps are showing and sometimes not after running the flow. Is there any permanent solution to get all the apps info other than the workaround of updating environment maker field as we never knew after which run the data will show correctly.

Sorry, I dont quite follow you. Can you please validate if you see the correct data in the entity itself by the export to Excel method described above?
The workaround I describe is just a way for you to instigate the flow for the environment of interest. That's not a thing you normally need to do.

Entity data is not showing correct data. Today after the schedule run , I have verified the entity data and it did not show all the apps, but after updating the environment maker field it is showing all the apps.
Seems during the schedule run the flow is not picking up all the MDA apps and if we update the maker field then the flow is picking up all the apps.

What do you mean by updating the maker field?
Also, can you see if there are failures in these runs?

Hi Jenefer,
Please find the below response
What do you mean by updating the maker field?
Followed the below steps for one of my environment for which MDA's are not showing correctly. After performing below steps the subsequent flow picked all the  apps.
Go in to the Environment entity in the Core solution
Got to Data > Select Environment of Choice > Edit > Edit Maker Name (you'll want to change back later) > Save
Go to Model Driven Apps flow and see it running (may take a few minutes to trigger)
can you see if there are failures in these runs?
Yes there were 2 failures but those were for different environments which I am not bothering at this moment.
Error 👍  ActionBranchingConditionNotSatisfied. The execution of template action 'Inquiry_not_supported_for_Teams_sku' skipped: the branching condition for this action is not satisfied.
Mainly I am looking for MDA's which are in Prod environment, which are not showing correctly during the schedule period(Although the flow ran successfully for this environment) and the apps are showing correctly after following the steps mentioned above(like editing Maker Name).

OK thank you. This means that the flow is working, and the data is correctly populating from the flows.
Again, we were only having you update that field in order to trigger the flow for your environment so that you didn't have to go and find it in the runs. So you triggered the flow, and it correctly populated all the data into Excel
Is it showing correctly in the dashboard now as well? And if not, can you please see which apps are missing?

Nope the flow is not working correctly as it is not picking up all the apps for all the environments, but when  I manually update  filed from Environment entity then only  the triggered flow is picking up all the MDA's.
To say in short the scheduled flow is not picking all the MDA's and those are not showing in Dashboard.

I'm sorry but I think you dont understand how the flows work. I was really just helping you re-trigger the flow in a way that wouldnt cause it to run for all environments. It was that same scheduled flow that you ran.
That flows is triggered when the envt record in the CoE is edited.
Normally it is edited by the main sync flow, but that causes all records to be edited making it hard to find your case.
We had you manually edit one record instead so that only that environment would run.
But it is the same flow.
Is the dashboard correct now?

I understand Jenefer. My only question is why the same flow which runs automatically after the main sync flow triggers shows incorrect data and why it is showing correct data after re-triggering the flow for particular environment.

I'm not sure it was incorrect. We are currently investigating.
Previously I believe you were not validating that they data was correct in the entity itself (by checking excel). And now we know that its correctly populating there. We were doing this to see if the problem was in the flow (aka the entity itself) or the report.
Can you please answer this question: is it showing correctly in the dashboard now?

I understand Jenefer. My only question is why the same flow which runs automatically after the main sync flow triggers shows incorrect data and why it is showing correct data after re-triggering the flow for particular environment.
As I said earlier the schedule flow did not show all apps in dashboard but when I re-trigger the flow for the environment which I am looking for picked up all the apps. The same data is visible in dashboard.
So for now can we please leave about the dashboard and look into the difference of scheduled flow and re-triggering for particular environment

I'm sorry I'm afraid I am unable to offer you the level of support you need here as it appears you need in person debugging in order to explore the issues you are seeing.
There are many partners that offer help with the CoE.