Instagram MPD DASH live stream - ExtractorError

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.12.23. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.12.23

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://instagram.ftxl1-1.fna.fbcdn.net/hvideo-ftw1/v/xxxxx-/live-dash/dash-hd/1111111.mpd?_nc_rl=abcdef&oh=ghij&oe=klmn']
[debug] Encodings: locale cp1252, fs utf-8, out cp1252, pref cp1252
[debug] youtube-dl version 2017.12.23
[debug] Python version 3.6.1 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg git-2017-12-23-9e5e323, ffprobe N-87867-g0655810072
[debug] Proxy map: {}
[generic] 1111111: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 1111111: Downloading webpage
[generic] 1111111: Extracting information
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "c:\program files\python34\lib\site-packages\youtube_dl\YoutubeDL.py", line 784, in extract_info
    ie_result = ie.extract(url)
  File "c:\program files\python34\lib\site-packages\youtube_dl\extractor\common.py", line 438, in extract
    ie_result = self._real_extract(url)
  File "c:\program files\python34\lib\site-packages\youtube_dl\extractor\generic.py", line 2181, in _real_extract
    self._sort_formats(info_dict['formats'])
  File "c:\program files\python34\lib\site-packages\youtube_dl\extractor\common.py", line 1069, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

This is the mpd that is on the above address:
<?xml version="1.0"?>

<MPD
  type="dynamic"
  xmlns="urn:mpeg:dash:schema:mpd:2011"
  xmlns:xsi="http://www.w3.org/2011/XMLSchema-instance"
  xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 DASH-MPD.xsd"
  profiles="urn:mpeg:dash:profile:isoff-live:2011"
  availabilityStartTime="2017-12-27T05:24:03-08:00"
  availabilityEndTime="2017-12-27T05:36:51-08:00"
  timeShiftBufferDepth="PT10S"
  suggestedPresentationDelay="PT2S"
  minBufferTime="PT1S"
  publishTime="2017-12-27T05:24:03-08:00"
  minimumUpdatePeriod="PT1S"
  publishFrameTime="0">
  <Period id="dash-hd" start="PT0S">
    <AdaptationSet
      segmentAlignment="true"
      maxWidth="396"
      maxHeight="704"
      maxFrameRate="30">
      <Representation
        id="live-hd-v"
        mimeType="video/mp4"
        codecs="avc1.4d401e"
        width="396"
        height="704"
        frameRate="30"
        startWithSAP="1"
        bandwidth="736441"
        FBQualityClass="sd"
        FBQualityLabel="396p">
        <SegmentTemplate
          presentationTimeOffset="0"
          timescale="1000"
          initialization="../live-hd-v/1111111-init.m4v"
          media="../live-hd-v/1111111-$Time$.m4v">
          <SegmentTimeline>
            <S t="260356" d="1000"/>
            <S t="261356" d="1000"/>
            <S t="262356" d="1034"/>
            <S t="263390" d="1000"/>
            <S t="264390" d="1033"/>
            <S t="265423" d="1000"/>
            <S t="266423" d="1033"/>
            <S t="267456" d="1000"/>
            <S t="268456" d="1000"/>
            <S t="269456" d="1000"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>
    <AdaptationSet segmentAlignment="true">
      <Representation
        id="live-hd-a"
        mimeType="audio/mp4"
        codecs="mp4a.40.2"
        audioSamplingRate="44100"
        startWithSAP="1"
        bandwidth="48563">
        <SegmentTemplate
          presentationTimeOffset="0"
          timescale="1000"
          initialization="../live-hd-a/1111111-init.m4a"
          media="../live-hd-a/1111111-$Time$.m4a">
          <SegmentTimeline>
            <S t="260356" d="1000"/>
            <S t="261356" d="1000"/>
            <S t="262356" d="1034"/>
            <S t="263390" d="1000"/>
            <S t="264390" d="1033"/>
            <S t="265423" d="1000"/>
            <S t="266423" d="1033"/>
            <S t="267456" d="1000"/>
            <S t="268456" d="1000"/>
            <S t="269456" d="1000"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

Description of your issue, suggested solution and other information
I tried downloading a dash mpd live stream from Instagram, as I was told yt-dl is able to dl mpd dash. I searched through the issues and some old ones say it's not possible, but other reports let me to believe you do now(?).
Anyway I get an ExtractorError for the mpd file above. I cannot give you an URL to test, as live streams only have a duration of a few minutes mostly.
If this cannot be solved with the information above, I could create a live stream and give you the DASH URL for you to test for some time via IRC.
Thank you

DASH livestreams are not supported.

What is the difference between dash live stream and dash download from YT?
Is it not the same downloading of segments and stitching them?
Will it be implemented in the future?
Thank you


Can not detect chrome for android

Thank you for your project,I tested a lot, It's "false" use "isMobile.android.phone" or "isMobile.phone" in chrome for android.thank you!

isMobile.other.chrome or isMobile.other.device


Creating jobs for testing purposes

I'm new to Go, so apologies if this is super-obvious.
I'm writing a Gearman worker using this library; I want to be able to test my functions, but I can't quite figure out how to create a test job to pass in to the method that actually does the processing.
Just as my super-basic example for now:
func DisplayMessage(job worker.Job) ([]byte, error) {
	log.Printf("DisplayMessage: Data=[%s]\n", job.Data())
	data := []byte(strings.ToUpper(string(job.Data())))
	return data, nil
}

I can see that I need to pass a struct of type worker.Job, but I can't figure out how to create that struct.
I guess if I needed to, I could use the client to send a job directly to Gearman, but I'd rather minimise the number of moving parts here. I couldn't see a test in the library that created jobs (but again, I might have missed that).
Can anyone help?

I think I've figured this out now, though not necessarily in the best way. Here's my test script:
package main

import (
	"testing"
	"github.com/mikespook/gearman-go/worker"
	"crypto/sha256"
)

type TestJob struct {
	JobHandle string
	JobId string
	JobData string
}

func (tj TestJob) Err() error {
	return nil
}

func (tj TestJob) Data() []byte {
	return []byte(tj.JobData)
}

func (tj TestJob) Fn() string {
	return "TestJob"
}

func (tj TestJob) SendWarning(data []byte) {
	// NOOP
}

func (tj TestJob) SendData(data []byte) {
	// NOOP
}

func (tj TestJob) UpdateStatus(numerator, denominator int) {
	// NOOP
}

func (tj TestJob) Handle() string {
	return tj.JobHandle
}

func (tj TestJob) UniqueId() string {
	return tj.JobId
}

func createJob(data []byte) (job worker.Job, err error) {
	hasher := sha256.New()
	hasher.Write(data)
	jobHandle := string(hasher.Sum(nil))

	jobData := string(data)

	jobId := jobHandle[0:16]

	newJob := TestJob{JobId: jobId, JobHandle: jobHandle, JobData: jobData}

	return newJob, nil
}

func TestDisplayMessage(t *testing.T) {
	expected := "HELLO"

	job, err := createJob([]byte("Hello"))
	if err != nil {
		t.Error("Couldn't create the job.")
	}

	actual, err := DisplayMessage(job)
	if string(actual) != expected {
		t.Errorf("Expected '%s', got '%s'", expected, string(actual))
	}
}

Is there an easier way to get to this? :)

It's not about "new to go", but "new to programing" ?
you don't need to call the func DisplayMessage,so you don't need make a Job, it's a callback function, just register (see https://github.com/mikespook/gearman-go/blob/master/example/worker/worker.go)
then send a command on another PC by Client.go to the gearman-server. if worker is running and is connected to the server,it could received the msg and call the func DisplayMessage

I think this project is dead .. no udpating, no issue closing

Hi @stevehill1981
You can't test the worker directly unless you write your own dummy-job and pass it to the callback.
But if you want to combine the testing of client and worker, you need Gearman server involved in.
It's not a good practical thing, isn't it? So, I abandoned Gearman from the architecture of my projects.
Hi @lanybass
Dying, but not dead.
The Gearman protocol has some bad design inside and personally, I've used MQTT to replace Gearman recent years.
The issue will keep open unless I think we have all the facts we need. Still, I will try to fix the significant issue but won't improve the library.
And If someone can relive this project, I'd like to transfer the project (or just make a clone and let me know).

Thanks @mikespook. I'm stuck with Gearman for the time being for legacy reasons, but perhaps, if I'm rewriting the workers anyway (moving from PHP to Go), I can justify changing the queueing system at the same time.
@lanybass - I wanted to test just the functionality of DisplayMessage; not the Gearman worker part of it. I'm coming from the PHP world (which may or may not make me a proper programmer, despite more than 10 years of experience), and when I was doing a similar thing there I had to extend the GearmanJob class so that I could set the appropriate data without having to run a full client/server setup. Apologies if this doesn't give me street cred as a real programmer.

I would highly recommend abstracting the functionality you use gearman for in your project to an interface. This way you can mock the gearman functionality in a unit test. Anything involving the actual gearman worker would be an integration test.


Error when php.ini's always_populate_raw_post_data is not set to -1

I checked this problem with versions 3.0 to 3.5, using PHP 5.6. When php.ini's always_populate_raw_post_data is not set to -1, these warnings/errors occour:

Deprecated:  Automatically populating $HTTP_RAW_POST_DATA is deprecated and will be removed in a future version. To avoid this warning set 'always_populate_raw_post_data' to '-1' in php.ini and use the php://input stream instead. in Unknown on line 0

Warning:  Cannot modify header information - headers already sent in Unknown on line 0

Warning:  session_start(): Cannot send session cache limiter - headers already sent in C:\...\index.php on line 3

Fatal error:  Uncaught exception 'RuntimeException' with message 'Unexpected data in output buffer. Maybe you have characters before an opening <?php tag?' in C:\...\vendor\slim\slim\Slim\App.php:552
Stack trace:
#0 C:\...\vendor\slim\slim\Slim\App.php(344): Slim\App->finalize(Object(Slim\Http\Response))
#1 C:\...\vendor\slim\slim\Slim\App.php(298): Slim\App->process(Object(Slim\Http\Request), Object(Slim\Http\Response))
#2 C:\...\index.php(170): Slim\App->run()
#3 {main}
  thrown in C:\...\vendor\slim\slim\Slim\App.php on line 552

It would be interesting to check the ini configuration in Slim\App.php and throw an exception with an appropriated message. Example:
$value = ini_get( 'always_populate_raw_post_data' );
if ( false !== $value && "-1" !== $value ) {
    throw new \RuntimeException( 'You must set the key "always_populate_raw_post_data" to -1 in your php.ini file before using Slim.' );
}

Additionally, to include this information in Slim documentation (maybe in the Getting Started section).

This problem isn't related to Slim. It's because php printed the Deprecated message before Slim send the response(causing the header error). You could just change PHP error report config on php.ini or:
error_reporting(E_ALL ^ E_DEPRECATED);

@mathmarques  Please try to comment the line always_populate_raw_post_data in your php.ini to see what happens.
The warnings are not the problem. This line is (Slim\App.php, line 552):
Fatal error:  Uncaught exception 'RuntimeException' with message 'Unexpected data in output buffer. Maybe you have characters before an opening <?php tag?' in C:\...\vendor\slim\slim\Slim\App.php:552

@mathmarques Adjusting the error reporting the way you informed also does the trick. However, when this report options is not set, the problem occurs. So, I think Slim could check for both, in order to inform about problems like these.

Slim isn't going to check for and provide a warning for every deprecated PHP ini directive.
The deprecation of always_populate_raw_post_data is in the migration guide: http://php.net/manual/en/migration56.deprecated.php

Of course not, and I didn't ask for this. But Slim could include some common problems in their checkings or, at least, in the documentation (Getting Started, FAQ, or whatever). This can save developers' time and avoid unnecessary Issues.

imho there should be a better error message.
I lost a lot of time to check files for "unexpected" characters.
by the way I tried all the workarounds
commented out or set to on|off|-1 ;always_populate_raw_post_data
and set error_reporting(E_ALL ^ E_DEPRECATED);
Nothing to do error still remains
I was an happy user of Slim 2.0 never had any problem.

@adriano-ghezzi This is related to a change in PHP 5.6 and not Slim.
In your php.ini file, you need to also remove the leading ; before always_populate_raw_post_data as that semi-colon means that your change doesn't have any effect. You also need to restart your web server when you change a php.ini setting.
The 'Unexpected data in output buffer.' exception is from Slim. To stop this happening, update your settings:
$config = ['settings' => [
    'addContentLengthHeader' => false,
]];
$app = new Slim\App($config);


Spatial Analysis Test

Okay, looks good. Can you make all of the filenames lowercase to follow the naming scheme with the rest of the repo? Also, add _utils to the filenames so that it follows the conventions of the other util scripts


CASMPET-6738: handle multiple ipxe pods

Description
Add the logic to move multiple ipxe pods that are possible in CSM 1.5 and beyond. For example, there could be old cray-ipxe pods from CSM 1.4 deployment; beginning in CSM 1.5, there could be new cray-ipxe-x86-64 and/or cray-ipxe-aarch64 pods that we want to move before rebuilding ncn nodes.

Checklist

[ ] If I added any command snippets, the steps they belong to follow the prompt conventions (see example).
[ ] If I added a new directory, I also updated .github/CODEOWNERS with the corresponding team in Cray-HPE.
[x] My commits or Pull-Request Title contain my JIRA information, or I do not have a JIRA.

/backport release/1.6


Update CVN token

Description

https://github.com/cosmos/chain-registry/pull/4511
Checklist

Adding Assets

If adding a new asset, please ensure the following:

[x] Asset is registered to the Cosmos Chain Registry.
[x] Add asset to bottom of zone_assets.json.

[x] chain_name and base_denom are provided and use values exactly as defined at the Chain Registry.
[x] path is provided, and the IBC channel referenced is registered at the Chain Registry (unless native to Osmosis).
[x] osmosis_verified is set to false
[ ] listing_date_time_utc is specified and accurate
[ ] Optional: transfer_methods, peg_mechanism, override_properties, canonical, categories, where necessary (see README for details).

Adding Chains

If adding a new chain, please ensure the following:

[x] Chain is registered to the Cosmos Chain Registry.

Chain's registration must have staking defined, with at least one staking_token denom specified.
Chain's registration must have fees defined; at least one fee token has low, average, and high gas prices defined.

[x] IBC Connection between chain and Osmosis is registered.
[x] Add chain to bottom of zone_chains.json

[x] rpc and rest does not have any CORS blocking of the Osmosis domain, and RPC node has have WSS enabled.
[x] explorer_tx_url correctly directs to the transaction when the hash is inserted into the URL.

Hi, @JeremyParish69

I checked the osmisis app and why the cvn assets shown above are not updated yet, is there anything else I need to do?

https://app.osmosis.zone/assets/CVN
https://app.osmosis.zone/api/edge-trpc-assets/assets.getAssetPrice,assets.getAssetPrice,assets.getUserMarketAsset,assets.getAssetPrice?batch=1&input=%7B%220%22%3A%7B% 22json%22%3A%7B%22coinMinimalDenom%22%3A%22uosmo%22%7D%7D%2C%221%22%3A%7B%22json%22%3A%7B%22coinMinimalDenom%22%3A%22ibc%2FD3FAF77F5DE21C18413B164751239 BA7D521A9D8EA53BFE553AADF338A721480% 22%7D%7D%2C%222%22%3A%7B%22json%22%3A%7B%22findMinDenomOrSymbol%22%3A%22CVN%22%2C%22userOsmoAddress%22%3Anull%7D%2C%22meta%22% 3A%7B%22values%22%3A%7B%22userOsmoAddress%22%3A%5B%22undefined%22%5D%7D%7D%7D%2C%223%22%3A%7B%22json%22%3A%7B% 22coinMinimalDenom%22%3A%22ibc%2F498A0751C798A0D9A389AA3691123DADA57DAA4FE165D5C75894505B876BA6E4%22%7D%7D%7D

In addition, can we create a liquidity pool as shown below?

Indeed I'd expect this to have updated. Raised the issue internally.

The fix has been published, and the denom looks correct to me now.


[Event Request] Codeunit 5985 Serv-Item Tracking Rsrv. Mgt. - TransferReservToItemJnlLine

Hi,
We would like to extend CU Serv-Item Tracking Rsrv. Mgt. TransferReservToItemJnlLine procedure like this:

[IntegrationEvent(false, false)]
    local procedure OnBeforeTransferReservToItemJnlLine(var ServiceLine: Record "Service Line"; var ItemJnlLine: Record "Item Journal Line"; var QtyToBeShippedBase: Decimal; var CheckApplFromItemEntry: Boolean; var IsHandled: Boolean)
    begin
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


CNRM unhealthy

ACM shows the following error.
kf-kcc-admin   KNV2010: unable to create resource: KNV2010: failed to create "iam.cnrm.cloud.google.com/v1beta1, Kind=IAMPolicyMember", "github-probots/jlewi-editor": Internal error occurred: failed calling webhook "iam-validation.cnrm.cloud.google.com": Post https://cnrm-validating-webhook.cnrm-system.svc:443/iam-validation?timeout=30s: no endpoints available for service "cnrm-validating-webhook"

The pods are unhealthy
kubectl --context=kf-kcc-admin -n cnrm-system get pods
NAME                                           READY   STATUS    RESTARTS   AGE
cnrm-controller-manager-0                      1/1     Running   0          7d7h
cnrm-deletiondefender-0                        0/1     Running   0          14d
cnrm-resource-stats-recorder-89dcfb5d8-cbhd6   0/1     Running   0          7d8h
cnrm-webhook-manager-577cb8d559-ckvrb          0/1     Running   0          14d
cnrm-webhook-manager-577cb8d559-pdnpx          0/1     Running   0          7d8h

Force delete all pods
kubectl --context=kf-kcc-admin -n cnrm-system delete pods --grace-period=0 --force --all=true

Pods are healthy again.
kubectl --context=kf-kcc-admin -n cnrm-system get pods
NAME                                           READY   STATUS    RESTARTS   AGE
cnrm-controller-manager-0                      1/1     Running   0          47s
cnrm-deletiondefender-0                        1/1     Running   0          47s
cnrm-resource-stats-recorder-89dcfb5d8-qz9bw   1/1     Running   0          47s
cnrm-webhook-manager-577cb8d559-f8289          1/1     Running   0          46s
cnrm-webhook-manager-577cb8d559-g2nn8          1/1     Running   0          46s


Render Models don’t show up in LWRP

Unity’s light weight render pipeline does not include the standard shader so render models don’t show up properly. See: https://www.reddit.com/r/SteamVR/comments/a0o5jp/vive_controllers_material_not_showing_in_unity/?st=JOZD7XK2&sh=cf9c5d12

What's particularly problematic is that the plugin contains custom shaders, e.g. for teleport highlight. And I have no idea how to update custom shaders for LWRP. :P

This is a killer; using the LWRP would seem to be a no brainer, as is using SteamVR.

This is a killer; using the LWRP would seem to be a no brainer, as is using SteamVR.

I completely agree, would be great to be able to use shadergraph to adjust shaders too.

Old issue, but is this now fixed with URP?

If URP has not fixed this, this issue is still open. It's absurd that it was never closed. LWRP is important for VR, as is Steam VR. This should have been a critically high issue.
Ah, maybe it's a duplicate of https://github.com/ValveSoftware/steamvr_unity_plugin/issues/422 but a comment on the closure would have been nice.


Linux + mesa + lime : shaders not working

Hello,
The now famous sample /git/samples/basic is not working with mesa.
As i use Linux, i'm using lime but i don't think it's related this time.
haxelib run lime test linux compiles well, but hangs at execution on a shader error.
I don't know if it's a mesa problem, or a problem inside the Intel's Xorg driver.
Depending on the codepath of the driver (i've tested with Intel haswell and skylake GPU), you get a cryptic error message about shader or nothing (empty string as error reason).
The cryptic error message was something about an invalid definition of struct mat3x4.
As struct is a keyword presents in all flavours of OpenGL core, i suspect something else.
By trial and errors, i've discovered that changing version of shader to GL-ES 1.00 made the sample working (as mesa supports GL-ES in 1.00).
So, i force Intel driver to use mesa-EGL (the gl-es part of mesa)
In basic_lime.xml
 <haxedef name="mesa" value="" />

And i've added the condition in hxsl/GlslOut.hx
 #if((cpp && mobile)||js||mesa) //added «mesa»
 decls.push("#version 100");
 #else
 decls.push("#version 130");

It's working with the above modifications.
BTW, it's working without modifications with nvidia binary driver.

What if you don't have #version ?
@kiroukou I think we should maybe add #version in GlDriver instead of GLSLOut since we can perform some runtime detection there

Totally agree.
I've let the comment for that purpose, but changing this is making a
dependency (requirement) between 2 modules (package) so i wasn't sure how
to do that correctly in the heaps way  :)
Le 14 mars 2016 20:15, "Nicolas Cannasse"<EMAIL_ADDRESS>a
écrit :

What if you don't have #version ?
@kiroukou https://github.com/kiroukou I think we should maybe add
#version in GlDriver instead of GLSLOut since we can perform some runtime
detection there
—
Reply to this email directly or view it on GitHub
https://github.com/ncannasse/heaps/issues/155#issuecomment-196482252.

@kiroukou I'm not sure I understand. You can either setup a global GLSLOut.VERSION static from your driver, so it output #version , or add it yourself in the driver once returned by hxsl. Hxsl should never reference the driver anyway

@ncannasse
If i don't push #version in GlslOut.hx , i get:
Error : An error occurred compiling the shaders: 0:1(1): error: syntax error, unexpected NEW_IDENTIFIER

precision mediump float;
struct mat3x4 { vec4 a; vec4 b; vec4 c; };
(...)

BTW, the error i get with #version 130 is:
Error : An error occurred compiling the shaders: 0:3(8): error: syntax error, unexpected MAT3X4, expecting IDENTIFIER or TYPE_IDENTIFIER or NEW_IDENTIFIER or '{'

#version 130
precision mediump float;
struct mat3x4 { vec4 a; vec4 b; vec4 c; };
vec3 m3x4mult( vec3 v, mat3x4 m) { vec4 ve = vec4(v,1.0); return vec3(dot(m.a,ve),dot(m.b,ve),dot(m.c,ve)); }

I don't see what is the problem, struct is a valid datatype even in glsl 130.
Intel driver is compatible with glsl 130 (cannot go above for the moment, but 130 is okay).
Tell me if i can do something more.

mat3x4 is now generated as _mat3x4 which should address the issue.


openai: add python3Packages.wandb dependency

Description of changes
Changes originally submitted in #161725, but since the changes will cause failed builds on darwin until #161740 lands on master (it's currently still only on staging), targeting staging with these changes instead.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@malob not sure why the tests fail

@SuperSandro2000, I don't know either, but it doesn't look like it has anything to do with the changes made by this PR, all 4 builds fail due to issues with dependencies (and on darwin pretty low level dependencies).
For x86_64-darwin this seems to be the issue:
[...]
checking run-time libs availability... failed
configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lnghttp2 -lssh2 -lssh2 -lssl -lcrypto -lssl -lcrypto -lgssapi_krb5 -lresolv -lz
builder for '/nix/store/j1xrg7iihy5hnhc69hjzr7x7wvd8h6sm-curl-7.82.0.drv' failed with exit code 1
cannot build derivation '/nix/store/hm5rfd4537jaij1ga0mgzhm7gl3xvynb-swift-corefoundation-unstable-2018-09-14.drv': 1 dependencies couldn't be built
[...]

On aarch64-darwin:
[...]
0.1.2/bin/sigtool --file "$file" check-requires-signature
Unexpected exit status from sigtool: 134
error: builder for '/nix/store/qh86d3a6a7wc5994zm1wklqcadq0fwki-python3-minimal-3.9.11.drv' failed with exit code 1;
       last 10 log lines:
       > /nix/store/q0vnphbkdcldkjrwp1xs69amf9843yjb-python3-minimal-3.9.11/lib/python3.9/config-3.9-darwin/makesetup
       > /nix/store/q0vnphbkdcldkjrwp1xs69amf9843yjb-python3-minimal-3.9.11/lib/python3.9/config-3.9-darwin/config.c
       > /nix/store/q0vnphbkdcldkjrwp1xs69amf9843yjb-python3-minimal-3.9.11/lib/python3.9/config-3.9-darwin/Setup.local
       > /nix/store/q0vnphbkdcldkjrwp1xs69amf9843yjb-python3-minimal-3.9.11/lib/python3.9/config-3.9-darwin/python.o
       > /nix/store/q0vnphbkdcldkjrwp1xs69amf9843yjb-python3-minimal-3.9.11/lib/python3.9/config-3.9-darwin/libpython3.9.a
       > dyld: Library not loaded: /nix/store/x7sy3ml65vg0h6kkkq01qmwqx5plpj8p-openssl-1.1.1n//libssl.1.1.dylib
       >   Referenced from: /nix/store/c04kqdcdfz3hrw3r2hi69lfmp6qk6mpw-sigtool-0.1.2/bin/sigtool
       >   Reason: image not found
       > /nix/store/p9v74za26sk90p313qmwmkfrm7w2sbx0-signing-utils: line 24: 37225 Abort trap: 6           /nix/store/c04kqdcdfz3hrw3r2hi69lfmp6qk6mpw-sigtool-0.1.2/bin/sigtool --file "$file" check-requires-signature
       > Unexpected exit status from sigtool: 134
       For full logs, run 'nix log /nix/store/qh86d3a6a7wc5994zm1wklqcadq0fwki-python3-minimal-3.9.11.drv'.
error: 1 dependencies of derivation '/nix/store/yx8vjnhsxq58919kr9cdl21nj6da5nam-ICU-66108.drv' failed to build
error: 1 dependencies of derivation '/nix/store/ylxrlp582pqvvi9dq72xik8pl0anjslf-libxml2-2.9.13.drv' failed to build
[...]

For Linux thing seem to get further. I'm going to have a look into those now.
I know that on darwin this change builds on master with #161740 cherry picked.

Looks like Hydra is having many of the same issues with the same dependencies on staging-next: https://malob.github.io/nix-review-tools-reports/nixpkgs:staging-next/nixpkgs_staging-next_1750665.html

Rebased to get fixes like 195c7da07df7311e9be87e660bf66fe7f8746169 which should help things.

Now wandb fails to build on staging...

@SuperSandro2000, just added a commit that fixes the python3Packages.wandb build issue on darwin systems.

Result of nixpkgs-review pr 165158 run on aarch64-darwin 1

6 packages built:

openai (python39Packages.openai)
python310Packages.openai
python310Packages.spacy-loggers
python310Packages.wandb
python39Packages.spacy-loggers
python39Packages.wandb

Result of nixpkgs-review pr 165158 run on x86_64-darwin 1

6 packages marked as broken and skipped:

python310Packages.spacy
python310Packages.spacy-transformers
python310Packages.textacy
python39Packages.spacy
python39Packages.spacy-transformers
python39Packages.textacy

6 packages built:

openai (python39Packages.openai)
python310Packages.openai
python310Packages.spacy-loggers
python310Packages.wandb
python39Packages.spacy-loggers
python39Packages.wandb

Result of nixpkgs-review pr 165158 run on aarch64-darwin 1
6 packages built:

How are you getting this result? When I run nixpkgs-review pr 165158 on aarch64-darwin it's starts trying to build hundreds (thousands?) of packages, including everything from coreutils to llvm. Am I missing something?

How are you getting this result? When I run nixpkgs-review pr 165158 on aarch64-darwin it's starts trying to build hundreds (thousands?) of packages, including everything from coreutils to llvm. Am I missing something?

That sounds right. I've been working on getting this building for a couple of weeks now, and have been running long builds in the background as I've been going to confirm the fixed I've submitted are working on staging/staging-next.


Native Notebooks - Add code cell not working after markdown

Steps to Reproduce:

Have a markdown cell followed by code cell
Try to add code cell using +code button between markdown and code cell

Actual Behavior
Markdown cell added, not code cell

This would better be addressed by supported languages info from active kernel and also some context in the document.

@rebornix I also saw this issue and opened it against jupyter as I only saw it in their notebooks. I'm not sure where this bug should live.

Fixed via https://github.com/microsoft/vscode/commit/2a1f20afc7c2a52a74806c88c21fc4e333c36363.


Generating Spectrogram as the rendered instead of the waveform.

Wavesurfer.js version(s): 6.2.0
Browser and operating system version(s): MacOS Monterey v 12.1
I dont know how to change the waveform rendered to show spectrogram instead of the waveform. Please help.

I'm not sure what you mean? can you add a screenshot or example code to explain the issue?

no feedback, closing.


Firebase Database not pruning tracked queries

Xcode version: 9.4.1 (9F2000)
Firebase SDK version: 5.4.0
Firebase Component: Database
Component version: 5.4.0

Description of the problem
The time it takes for the first callback to occur when observing a reference increases proportionally with the number of writes made. In some cases this means it takes nearly three minutes for the first callback to occur.
Note: Turning off persistence fixes the issue.
Steps to reproduce:
Enable persistence.
Observe a database reference
Make thousands of writes
Re-open the app, the time taken before the first callback of the reference will have increased
I profiled the app in instruments and narrowed down the cause to FTrackedQueryManager.initWithStorageEngine
It loops through all tracked queries, in my case I had some 200,000 tracked queries. Looking at them in the debug viewer I noticed most were from writes which were made months ago.
It appears as though every time a write occurs a tracked query is added for the path of the write and is never removed, so overtime these tracked queries build up, leading to an ever increasing start up time for the FTrackedQueryManager.
There are functions to purge the cache but these are not public and they don't appear to be invoked when the Database is initialised.
Relevant Code:
Database.database().isPersistenceEnabled = true

@mikelehen the repro steps seem pretty ordinary, I'm surprised this hasn't shown up before. Is this a regression?

This is probably related to https://github.com/firebase/firebase-ios-sdk/blob/c6b4b03fffc3cea7c9525e5c79dce28f52900521/Firebase/Database/Persistence/FPersistenceManager.m#L124
When you perform a write we add it to our cache, which includes adding it to our tracked queries (which helps us know that we have complete data for that location and can raise events for it if you listen in the future).
I believe our cache policy tries to keep only up to 1000 tracked queries (https://github.com/firebase/firebase-ios-sdk/blob/c6b4b03fffc3cea7c9525e5c79dce28f52900521/Firebase/Database/Persistence/FCachePolicy.m#L26) but we only check whether to prune the cache after every 1000 listen updates (see the line above).  So if your app only does writes (or mostly writes with very few listens or very infrequently-changing listens) then we may never attempt to prune the cache, allowing the tracked queries to grow...
A simple improvement may be to add a call to doPruneCheckAfterServerUpdate (https://github.com/firebase/firebase-ios-sdk/blob/c6b4b03fffc3cea7c9525e5c79dce28f52900521/Firebase/Database/Persistence/FPersistenceManager.m#L153) from applyUserWrite (https://github.com/firebase/firebase-ios-sdk/blob/c6b4b03fffc3cea7c9525e5c79dce28f52900521/Firebase/Database/Persistence/FPersistenceManager.m#L123). That way we'd do the prune check after each write and should actually limit you to ~1000 tracked queries.  [though we also should do some renaming of afterServerUpdate or at least add comments to explain what's going on]
@schmidt-sebastian officially maintains the Realtime Database SDK though I don't know if he has cycles to look into / implement / test this.  If you were so inclined, providing a PR might be more likely to get this fixed in the near-term. :-)

Thanks @mikelehen for the speedy response. @schmidt-sebastian do you think you'll be able to have a look? If not I'd be happy to have a go at fixing it :)
Thanks!

I can take a look at this next week or so. If you want to take a stab at it in the meantime, I would be more than happy to support you along the way. Thanks!

Some more info...
After a write FPersistenceManager.doPruneCheckAfterServerUpdate is executed which checks to see if a prune is required according to its cache policy pacing in its serverCacheUpdatesSinceLastPruneCheck value.  However as the serverCacheUpdatesSinceLastPruneCheck is not persisted between app restarts, a prune will only occur after 1000 server updates during a single app session.
Some options:

Always perform a prune check (it doesn't appear to be a very CPU intensive task)
Reduce the maximum number of server updates before performing a prune
Persist the number of server updates since last prune check between app sessions
On first server update of any app session, always perform a prune

What do you suggest @schmidt-sebastian

I would suggest you pick option 4 and perform a prune check at startup, and then once every 1000 operations. Since we target a wide range of mobile devices, it is sometimes very hard to predict what the perceived performance characteristics of changes in the persistence logic are.

doPruneCheckAfterServerUpdate is actually pretty expensive because it has to iterate the entire database to estimate the cache size. It ends up calling https://github.com/firebase/firebase-ios-sdk/blob/master/Firebase/Database/third_party/Wrap-leveldb/APLevelDB.mm#L289
So #4 is the most palatable of your options but it'll still slow down app starts for everybody.  And it'll still only help apps once they restart.  During the life of the app, if they're only doing writes (and not getting server updates) they'll never do another prune.
And so I still think my suggestion in https://github.com/firebase/firebase-ios-sdk/issues/1518#issuecomment-404264214 probably makes the most sense.

Thanks for the input.
@mikelehen If doPruneCheckAfterServerUpdate is expensive then would it not be a bad idea to be calling it inapplyUserWrite? In some cases our app might make multiple writes per second.
Could another option be to add a public method to perform a prune? Then users can perform it when it best suits the needs of their app. For example, I'd probably choose to do it after start-up and after an initial fetch from the database, or maybe when the app is sent to the background.

@marcbaldwin Well, because of the kFServerUpdatesBetweenCacheSizeChecks counter in there it would still only do the expensive part of the check once every 1000 writes...  so you might see the occasional delayed write, but it shouldn't be too bad...  hopefully. :-)
Adding an API would certainly provide the most control for developers, but 1) ideally we'd solve this transparently for the majority of use cases without developers needing to change their code, and 2) adding an API is frankly more time-consuming and expensive. The Firebase APIs are carefully curated and there's an internal design / approval process that we'd need to go through to make changes.  It's certainly doable, but it's not something to be taken too lightly.

@mikelehen @schmidt-sebastian sorry for the delayed response, I've been away on holiday.
@mikelehen calling doPruneCheckAfterServerUpdate from applyUserWrite still won't help unless our app performs 1000 writes or we change the if statement to account for the number of tracked queries.
// Current
- (void)doPruneCheckAfterServerUpdate {
    self.serverCacheUpdatesSinceLastPruneCheck++;
    if ([self.cachePolicy shouldCheckCacheSize: self.serverCacheUpdatesSinceLastPruneCheck]) {

/// proposed
- (void)doPruneCheckAfterServerUpdate {
    self.serverCacheUpdatesSinceLastPruneCheck++;
    if ([self.cachePolicy shouldCheckCacheSize: self.serverCacheUpdatesSinceLastPruneCheck
                numberOfTrackedQueries: self.trackedQueryManager.numberOfPrunableQueries]) {

Thanks!

@marcbaldwin Hrm.  Are you saying that your app is not doing more than 1000 writes?  From your original statement about having 200,000 tracked queries, my assumption was that the pruning would kick in every time you reached ~1000, which should be sufficient to keep the performance good.
self.trackedQueryManager.numberOfPrunableQueries is an O(n) operation (it requires walking all of the tracked queries) and so doing it after every write or server update could impact performance significantly.
It also wouldn't help unless we also changed our current cache policy, since kFMaxNumberOfPrunableQueriesToKeep is set to 1000 (https://github.com/firebase/firebase-ios-sdk/blob/1d484d33a778a9778a728829a1e30dded4a3d68c/Firebase/Database/Persistence/FCachePolicy.m#L26).

Out of curiosity, how many app sessions does it take for your app to hit 200,000?  I assumed there must be some action in your app that triggers a large number of writes and hopefully it'd (at least periodically?) exceed 1000 and the pruning would kick in.  It's not foolproof but I would expect it'd be good enough for most apps.
But yeah, persisting serverCacheUpdatesSinceLastPruneCheck would be more foolproof, dealing with the case of an app that accumulates many tracked queries over time but only a small number each app session. But I'm somewhat wary of making schema changes.  Most of our development effort is going into Cloud Firestore and we only have limited bandwidth to implement / test features for Realtime Database. Schema changes are more risky and time-consuming. :-/
Also FWIW, persisting an extra value on every write / server update would make persistence a little slower...  so if most apps don't need it, it may not be the right trade-off.

Depends on the user, but a few months most likely. Thing is even with 50,000 tracked queries it's still ~30 seconds of start-up time.
Anyway, if we perform the prune check on writes, I could just put a dirty hack in to our app to perform 1000 writes for users who are experiencing an incredibly slow start-up time.
I'll make a PR for the call to doPruneCheckAfterServerUpdate from applyUserWrite if you're happy with that?

So thinking about this fresh today, I have a new suggestion (a tweak of previous things discussed above).
I don't want to call self.trackedQueryManager.numberOfPrunableQueries after every server update / write, but it would probably be fine to call it once at startup since startup already did the O(n) work to load the trackedQueries from persistence...  but I don't want to do the serverCacheEstimatedSizeInBytes work because that really is too expensive to do on startup.
So I suggest:

We extract lines 156 to 171 of doPruneCheckAfterServerUpdate into a new doPruneCheck method.
On startup (in initWithStorageEngine), we add code to check only the numberOfPrunableQueries, e.g.:

// Check if we have enough prunable queries to warrant doing a prune check on startup
// (but don't do the expensive work of estimating the cache size).
if ([self.cachePolicy shouldPruneCacheWithSize:0
        numberOfTrackedQueries:self.trackedQueryManager.numberOfPrunableQueries]) {
  [self doPruneCheck];
}

What do you think?

That sounds good to me. Are you going to do this or shall I?

If you don't mind putting together a PR, that'd be great.

@mikelehen trying out your suggestions.
If a prune is performed on start-up, the start-up time is just as bad as it was before! (I think it's O(n) where n is the number of items in the Database). So if during an app session there are >1000 writes/reads, they'll always get poor startup performance!
Looking at the prune function you can see it's incredibly inefficient (someone even left a TODO there). It scans the entire DB, and for each entry traverses the prune tree to see if it should prune it!
If we're going to prune on start-up, we need to optimise the prune function too. Is it possible to iterate the forest and remove from the DB? Or any other suggestions?
Thanks again
FLevelDBStorageEngine.m

- (void)pruneCache:(FPruneForest *)pruneForest atPath:(FPath *)path {
    // TODO: be more intelligent, don't scan entire database...

    __block NSUInteger pruned = 0;
    __block NSUInteger kept = 0;
    NSDate *start = [NSDate date];

    NSString *prefix = serverCacheKey(path);
    id<APLevelDBWriteBatch> batch = [self.serverCacheDB beginWriteBatch];

    [self.serverCacheDB enumerateKeysWithPrefix:prefix usingBlock:^(NSString *dbKey, BOOL *stop) {
        NSString *pathStr = [dbKey substringFromIndex:prefix.length];
        FPath *relativePath = [[FPath alloc] initWith:pathStr];
        if ([pruneForest shouldPruneUnkeptDescendantsAtPath:relativePath]) {
            pruned++;
            [batch removeKey:dbKey];
        } else {
            kept++;
        }
    }];
    BOOL success = [batch commit];
    if (!success) {
        FFWarn(@"I-RDB076021", @"Failed to prune cache on disk!");
    } else {
        NSTimeInterval time =  [start timeIntervalSinceNow]*-1000;
        FFDebug(@"I-RDB076022", @"Pruned %lu paths, kept %lu paths in %fms", (unsigned long)pruned, (unsigned long)kept, [start timeIntervalSinceNow]*-1000);
    }
}

Oh, yes. pruning is O(n) and that's probably not avoidable... at least not without a rewrite which is unlikely to happen.
But if we prune on startup, then you should stay under 1000 queries, which should be pretty fast to prune, right?  In other words, yes you'll have O(1000) startup delays occasionally, but you'll never reach O(200k) like you're hitting right now.
Or are you saying that occasionally pruning at 1000 queries is too slow to accept?

Yeah because pruning is O(number of entries in database) NOT O(number of prunable queries) it takes a loooooong time if your database has a lot of entries!

Typically "number of entries in database" is proportional to "number of tracked queries" but if you have persisted queries with many results, there could be a lot of entries.
I am indeed getting scared that we can't make this change...  but I'm curious what kind of delays you're seeing for your use case.

If a prune happens I was getting a delay of around 15 seconds for 1001 tracked queries, our app accumulates tracked queries pretty quickly, just not in one session, so perhaps every few sessions we'd get a start up time of around 15 seconds.
Do you guys have any time to look into this?

Unfortunately probably not. :-(  It sounds like we haven't landed on a straightforward solution that'll meet your needs nicely and right now most of our SDK development effort is going into the new Firestore SDKs rather than Realtime Database, so we're not able to invest in large or risky changes.
At this point if you can come up with a solution that works for you, you could consider working off a fork of the codebase.  For instance if you don't care about these writes being automatically stored in the offline cache, you could just make https://github.com/firebase/firebase-ios-sdk/blob/c6b4b03fffc3cea7c9525e5c79dce28f52900521/Firebase/Database/Persistence/FPersistenceManager.m#L123 into a no-op.  From my understanding, this would prevent the pruned queries and cache from growing.
Sorry I don't have a better answer right now.

Closing due to inactivity.


Plugin install fails with 404 (Not Found) error

❯ yarn dlx yarn-plugin-auto-install
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0013: │ yarn-plugin-auto-install@npm:1.9.1 can't be found in the cache and will be fetched from the remote regis
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0000: └ Completed
➤ YN0000: Done with warnings in 0s 134ms

$ yarn plugin import https://github.com/WillBooster/yarn-plugin-auto-install/releases/download/v1.9.1/index.min.cjs
➤ YN0000: Downloading https://github.com/WillBooster/yarn-plugin-auto-install/releases/download/v1.9.1/index.min.cjs
➤ YN0035: The remote server failed to provide the requested resource
➤ YN0035:   Response Code: 404 (Not Found)
➤ YN0035:   Request Method: GET
➤ YN0035:   Request URL: https://github.com/WillBooster/yarn-plugin-auto-install/releases/download/v1.9.1/index.min.cjs
➤ YN0000: Failed with errors in 1s 159ms

Looks like v1.8.7 has this file but v1.9.0 doesn't

@scowalt Thanks! I've fixed the issue and make yarn dlx yarn-plugin-auto-install work well.

Thanks! 🙌

This might be broken again with v2.0.0?
% yarn dlx yarn-plugin-auto-install
➤ YN0000: · Yarn 4.0.2
➤ YN0000: ┌ Resolution step
➤ YN0085: │ + yarn-plugin-auto-install@npm:2.0.0
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed
➤ YN0000: · Done in 0s 40ms

$ yarn plugin import https://github.com/WillBooster/yarn-plugin-auto-install/releases/download/v2.0.0/index.cjs
➤ YN0000: Downloading https://github.com/WillBooster/yarn-plugin-auto-install/releases/download/v2.0.0/index.cjs
➤ YN0035: The remote server failed to provide the requested resource
➤ YN0035:   Response Code: 404 (Not Found)
➤ YN0035:   Request Method: GET
➤ YN0035:   Request URL: https://github.com/WillBooster/yarn-plugin-auto-install/releases/download/v2.0.0/index.cjs
➤ YN0000: Failed with errors in 0s 335ms

@bradleyayers Sorry for the inconvenient. I've fixed the issue. Could you try again?

Works great, thanks!


UAT: Clean up master data to remove unwanted values from DIGIT core platform

Following master tables need to be cleaned up
common-masters.Department
common-masters.Designation
egov-hrms.EmployeeType
egov-hrms.Specalization
egov-hrms.EmploymentTest
egov-hrms.Degree
egov-hrms.DeactivationReason
Schema defined but no data. Hence remove schema definition
common-masters.Uom
common-masters.UomCategory
case.CaseType
Not clear what the following schemas are for. Says something about "Trade type for trade license applications".
Pucar.SummonOrder
Pucar.SummonOrder10
@subhashini-egov @suresh12 @Ramu-kandimalla @Susmitabe  - will suggest that this clean should be taken up in sprint 12 itself

@Ramu-kandimalla We cannot go live with incorrect data.
@suresh12  FTI. Should get addressed before go live

court related master data for department, designation etc is documented here  https://docs.google.com/spreadsheets/d/1FnqPw9iBYr12K-eMY-urPC2t7AHZvw1ceORkSPekQWw/edit?gid=1246655726#gid=1246655726
@Taherabharmal  can you please provide correct values

branch name : - cleaned-mdms-dev

Remove the RAINMAKER-PGR folder and associated masters. Remove from DB also. Else, this looks clean. @prasanna2400 @Susmitabe . Ensure you clean up the DB tables also - remove schema + data for these masters.

Hi @prasanna2400 Please merge this to DPG Dev and QA team will verify @rajeshcherukumalli

Hi @Ramu-kandimalla , verified the same with @prasanna2400  in DPG-DEV environment working fine will move the ticket to UAT release.

@suresh12 @Ramu-kandimalla we are still having incorrect data in UAT and that has been pushed to PROD. Need to address this ASAP
cc. @subhashini-egov

@atulgupta2024 I've documented what's present in UAT/prod here. Called for a meeting tomorrow, Dec 5 to discuss. Let's track this with this sheet. CC: @suresh12

@subhashini-egov i have added another sheet with the data dump also -- Master Data in SDC Production 1.0
cc. @suresh12


error[E0463]: can't find crate for miniz_oxide

https://github.com/ivmarkov/rust-esp32-std-demo/runs/5270253793?check_suite_focus=true
error[E0463]: can't find crate for `miniz_oxide`
   --> /home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/lib.rs:369:1
    |
369 | extern crate miniz_oxide;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

For more information about this error, try `rustc --explain E0463`.
error: could not compile `std` due to previous error

https://github.com/rust-lang/rust/pull/94220


Add clue text for stealing from ardougne chest after SotE completion …

closes #10854
…(spoiler alert)
Details: https://github.com/runelite/runelite/issues/10854
I haven't completed Song of the Elves but apparently a certain castle changes ownership, and the clue text to steal from its chest is changed accordingly.

place closes #<issue_number> in the pr description to link the pr to that issue and have the merge auto close it.
also, i think just the text of the current clue changed instead of a totally new one being added.

The wiki still has the old one listed, so it might still be valid? Does anyone know?

The wiki still has the old one listed, so it might still be valid? Does anyone know?

Shall I re-add the old clue text to be safe? Let me know

If noone has seen a Steal from a chest in King Lathas' castle in East Ardougne. then the wiki is probably just wrong and we should edit that. Especially if you haven't done sote yet and still see it.

@BrandtHill Did you get this clue on your account which hasn't completed SotE?

@Nightfirecat No, just cutting teeth on open source and went off what the linked issue said

I guess we can try this and see if we get any complaints.


Reasons for non-reproducibility

Hi, thanks for your work again.
Your previous version of code can not achieve the mAP you claimed, but your current version reached.
I guess the main reason for this difference is due to the different ways to process VOC2007 annotations.
Note that the evaluation of VOC2007 object challege excluedes the object marked difficult and this can be found in their  paper.
In your current version, you used the coco format annotation and code, which excludes the object marked difficult (see ignore attribute in json files and __add_gt_annotations function in json_dataset.py), while in your previous version (well organized one), you directly processed the annotation file and parsed the annotation file, and hence did not ignore the difficult objects.
In your current version, I annotated the code about ignoring the difficult objects and reran the code. A 47 mAP was achieved which is similar to your previous code.
I am not sure you can achieve the mAP reported in your paper after you modifiying the code in your well-organized version, but I think this is a big improvement and look forward to your well-organized version.

Hi, good point!  I forgot about the dificult guys. :facepalm:
I think that there are more problems in the refactored code and because that you where not able to reproduce the results.
Thanks :)


Allow composition of themes

If this worked, it would be really powerful:
theme_base = Theme(line_width=4pt, default_point_size=7pt)
theme_green = Theme(theme_base; default_color=colorant"green")
theme_orange = Theme(theme_base; default_color=colorant"orange")
theme_small_orange = Theme(theme_orange; line_width=1pt)

I'm imagining it as the four themes being equivalent to these:
theme_base = Theme(line_width=4pt, default_point_size=7pt)
theme_green = Theme(line_width=4pt, default_point_size=7pt, default_color=colorant"green")
theme_orange = Theme(line_width=4pt, default_point_size=7pt, default_color=colorant"orange")
theme_small_orange = Theme(line_width=1pt, default_point_size=7pt, default_color=colorant"orange")

Agreed.  I often want to tweak a single setting in a base theme for a specific plot and end up overwriting all the other settings.

Could you all elaborate a bit more on this suggestion?

Say I have a theme which I use for several graphs with a bunch of options (using the example above):
theme_base = Theme(line_width=4pt, default_point_size=7pt)

Then, if I want to change one setting on a single graph what I would like to do is something like
theme_green = Theme(theme_base; default_color=colorant"green")

However, this throws an error.  Instead you have to specify every singe option over again and construct a new theme:
theme_green = Theme(line_width=4pt, default_point_size=7pt, default_color=colorant"green")

(This is a moot point for me as I no longer use Gadfly, just expanding on the original issue).

Ah, I missed the theme_base; in the Theme parameters. Thanks for making this clear. Out of curiosity, what do you use now?

I use @spencerlyon2's excellent PlotlyJS.jl.  I use the normal Plotly javascript library for my web-based plots and it's very handy to also use the same package in Julia via Jupyter.

Hey @amellnik thanks for the shout out.
Coincidentally I'm actually planning on adding this exact feature to PlotlyJS.jl within the next week!


limit to zoom to horizontal (x) or vertical (y)

Based on #37, I see how to easily limit the panning to one axis.  However, zooming does not offer this functionality.  Would it be possible to limit the zoom easily to one axis?  Although this is not the expected application, but maybe something like a timeline where we would like to zoom to a particular subperiod.
Related to https://github.com/timelyportfolio/svgPanZoom/issues/8

I believe that it is possible to be done now. There are 2 different features that should allow that:

you can return false from beforeZoom and that zooming will get canceled
you can dynamically update min/maxZoom via setMaxZoom and setMinZoom

Can you please explain in more detail how limiting zooming on one axis should work? It is just hard for me to understand how you can limit zoom on one axis because zooming is done for both axis in parallel.

I'm closing this issue until more details are provided.


feat: add role to credential and proof record

Continues #1438

Not entirely sure why tests are failing on the agent.config...
https://github.com/openwallet-foundation/credo-ts/actions/runs/7958832675/job/21724520202?pr=1764#step:5:675

Might need some help with setting up new snapshots :).

Might need some help with setting up new snapshots :).

Did you already address this? Anything I can help with?

Should be ready now @berendsliedrecht


factorial, coils

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 5
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[c,f]

/trunk merge


Added Transaction context passing to turrets

This PR comes after #51 and with https://github.com/TheGhouls/oct-turrets/pull/14
As said in PR 14,
It helps to create something from HQ and give dynamic data to turrets.
With this set of patch, we could imagine the following scenario:

A HQ in its setup build things and keep it in its self.transaction_context
Before start each turret, it sends these data with ZMQ set_transaction_context
Turret receive and store in self.transaction_context
When start cannons, it pass it and cannons store in their self.transaction_context
Each transaction receive and store it in self.context
Transaction can use data from HQ in its run

From 4. codebase already can do it.
I think the name transaction_context could be innapropriate because this data will pass and can be modified at each step of OCT's mechanism. But at the end it is for transactions.

Coverage increased (+0.2%) to 93.947% when pulling d1ca1437b072524af28dc248c6736853cbe3b1b0 on ZuluPro:trans_context into c433b6dd0a76112e583dc5820a146d8750d2bcef on TheGhouls:master.


DLP-810: Add support for DLP payload logging

Adds support for the DLP payload logging feature, both enabling it for an individual rule and setting the account-level public key setting. Both new options can be added into existing resources. While you need an entitlement to set the public key, you don't need one to read it. Setting the key isn't attempted unless the public_key {} block is specified on the cloudflare_teams_account resource, so this shouldn't be a breaking change.

acceptance tests all looking good
TF_ACC=1 go test ./internal/sdkv2provider -v -run "^(TestAccCloudflareTeamsAccount|TestAccCloudflareTeamsRuleBasic)" -timeout 120m -parallel 1
=== RUN   TestAccCloudflareTeamsAccountConfigurationBasic
--- PASS: TestAccCloudflareTeamsAccountConfigurationBasic (18.26s)
=== RUN   TestAccCloudflareTeamsRuleBasic
--- PASS: TestAccCloudflareTeamsRuleBasic (12.74s)
PASS
ok  	github.com/cloudflare/terraform-provider-cloudflare/internal/sdkv2provider	31.448s

thanks @OrionNebula! you rock 🤘

Hi @jacobbednarz and @OrionNebula, this pull request seems to introduce a regression for the resource type cloudflare_teams_account.
I do not have DLP payload logging enabled and I am not providing any DLP-related configuration to this resource but I am faced with the following error: `Error: error finding DLP Account config "<account_id>": not found (3307).

Hi @rodmatos, thanks for pointing this out and sorry for the trouble! I think this is due to the fact that the payload log settings endpoint will return 404 if you've never configured it before, but there's no check for this case in the read function for the resource. I'll try to have a fix for this today.

Super, thank you so much @OrionNebula, let me know if you need some request logs or anything else from my side :-)


Costs section: kind returns empty string if alias not registered

Now returns value searched for.

resolved by making the toDisplatName filter return the key if corresponding value not found.


downloading from send_file unpacks gzipped files

The gzipped files returned with
return send_file( fpath, as_attachment=True )

are being downloaded unpacked. The file are saved as *.gz but its content is plaintext.
This does not happen with
Flask==1.1.2
werkzeug==1.0.1
Environment:

Python version: 3.7.1
Flask version: 2.0.1

I have the same issue.
I found that doing:
curl -X GET http://myurl.com/files/1 --output /tmp/test.tar.gz

works fine, and the filesize is ~10kb
But when I do:
import requests
resp = requests.request('GET', url='http://myurl.com/files/1')
with open('/tmp/test.tar.gz', 'wb') as f:
    f.write(resp.content)

The file size is ~100kb
I am using the following very simple call in my flask API:
import io
from flask import send_file

send_file(
    io.BytesIO(open('/src/test.tar.gz', 'rb').read()),
    download_name='test.tar.gz',
    as_attachment=True,
)

Yes, @AbdealiJK - downloading with curl is OK - ie.
    curl -O "http://<IP_ADDRESS>:5000/download/whatever.gz"

Hey all, not sure if you saw my comment above, but anyone is welcome to submit a PR to fix this.

So maybe send_file should only set Content-Encoding if as_attachment is False?

I can confirm that the suggested change works as expected:
      #if encoding is not None:
      if encoding is not None and not as_attachment:
          headers.set("Content-Encoding", encoding)

@davidism I have create a PR with the suggested changes:
https://github.com/pallets/werkzeug/pull/2161
Could you please review it?


Removed not used include files

Removed not used include files:

migration-guide include files are not considered
An include file is considered to be not used if the docs folder doesn't contain a file that contains includes/<include-filename> substring

Related to #4801

This PR doesn't appear in the build report. Closing and reopening to begin new build.


Lighting for ViroFlex

Are there any examples of how to apply a spotlight to a ViroFlex object?

Hi @rafthedeveloper,
The default lighting model for every component is 'Constant' for all objects in the scene, which will light the object evenly. To make the component respond to lights you'd have to add a material that defines the lighting model of the component.
To light a ViroFlexView,  you'll need to add a material to the ViroFlexView and its children so it can receive the light. Sample below:
An example in your render function will look like:
  <ViroSpotLight
            innerAngle={5}
            outerAngle={40}
            direction={[0,0,-1]}
            position={[0, 0, -.5 ]}
            color="#ff0000"
            castsShadow={true}
            shadowNearZ={.1}
            shadowFarZ={6}
            shadowOpacity={.9}
            intensity={1500}
            attenuationStartDistance={5}
            attenuationEndDistance={10}
          />
        <ViroFlexView materials="test_material" style={styles.containerHorizontal} position={[-2, 0, -3]} width={3} height={2} transformBehaviors={"billboard"} >
          <ViroImage materials="test_material" style={{flex:1}} source={{uri: "http://wiki.magicc.org/images/c/ce/MAGICC_logo_small.jpg"}} />
          <ViroImage materials="test_material" style={{flex:1}} source={{uri: "http://wiki.magicc.org/images/c/ce/MAGICC_logo_small.jpg"}} />
        </ViroFlexView>

The materials will be defined after your class like following:
ViroMaterials.createMaterials({
  test_material: {
    shininess: 2.0,
    lightingModel: "Blinn",
    diffuseColor: "#ffffff"
  }
});

The above assigns the material 'test_material' to each component to the ViroFlexView so it responds to the light. The lightingModel that'll be used is Blinn and the material color is white #ffffff.
You can learn more here about our lighting models here:
https://docs.viromedia.com/docs/3d-scene-lighting
Let me know if you have any more questions. Thanks!


Modify the AUTHOR section

Has this bug been raised before?

[X] I have checked "open" AND "closed" issues and this is not a duplicate

Description
The current "AUTHOR" section in the README does not clearly present the contributors or lacks certain relevant details. The section could be enhanced to provide a more organized and visually appealing display of the author(s) with additional context if needed.
Steps to Reproduce
Relevant Details

What page you were on: README file in the repository.
What you were trying to do: Understand the contributors' information in the AUTHOR section.
What went wrong: The section appeared cluttered or lacked visual clarity, making it difficult to quickly identify contributors.

Screenshots
N/A
Do you want to work on this issue?
Yes
If "yes" to the above, please explain how you would technically implement this.
Steps to Reproduce

Open the README file of the repository.
Scroll down to the "AUTHOR" section.
Observe the current format and content.
Notice the areas that could be improved for clarity or visual appeal.

Expected Behavior

A more organized presentation of the AUTHOR section.
Visual elements like images or badges could be added.
Clear and concise details about each author or main contributor.
Links to the contributors' profiles for better accessibility.

@rohitinu6 assign me the task under GSSOC ext 24 And HACKTOBERFEST. (LEVEL 2 if possible)

@alo7lika this issue is quite similar to the PR #182 , which you have contributed, please do not spam.
Try making changes in this pr, will assign the level accordingly


Can't read "large" files

Description
The editor consistently crashes while loading files larger than 3MB.
I am suspecting this has to do with the way the file is loaded or all of the different modules applied to it, resulting in rather large data sets and/or operations.
I would suggest that if a larger file is detected, it either switch to using a streaming file load and only loading what is currently seen or that automatic markup/module runs and what not are automatically disabled for large files.
3MB is a rough estimate of about 3-4000 lines of code, so this would also essentially cause trouble with writing larger blocks of code or larger libraries.
Steps to Reproduce
Load a file larger than 3MB
Expected behavior: Open the file for viewing
Actual behavior: Editor hangs or crashs
Versions
1.10.2

This is a duplicate of #8864 - feel free to subscribe there for updates.


feat: support jsdoc @example parse

Generate jsDoc @example content #28

Super cool!
Merged and released in v0.6.0. Thanks! 💪


feat: add notation plugin uninstall command

This PR is based on the spec PR: https://github.com/notaryproject/notation/pull/809
It adds the notation plugin uninstall command.
Tests: The e2e test cases are also included in this PR.
Note: This PR depends on the notation-go PR: https://github.com/notaryproject/notation-go/pull/363
Update: PR https://github.com/notaryproject/notation-go/pull/363 has been merged.

This PR was reviewed here: https://github.com/notaryproject/notation/pull/832, however, due to some GitHub issue, I accidently deleted the original branch and overwrite it with the current one (the contents of these two PRs are the same).


Add linters for ROS1

Issue #, if available:
Description of changes:
Add linters for ROS1
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

Looking at the logs, I only see .py files are checked, e.g. under flake8
 2 files checked
No problems found

Checked files:

* robot_ws/src/hello_world_robot/setup.py
* robot_ws/src/hello_world_robot/src/hello_world_robot/__init__.py

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

Looking at the logs, I only see .py files are checked, e.g. under flake8
 2 files checked
No problems found

Checked files:

* robot_ws/src/hello_world_robot/setup.py
* robot_ws/src/hello_world_robot/src/hello_world_robot/__init__.py

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

We are also missing files like this, because they don't have a .py extension even though they are Python scripts.

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

We are also missing files like this, because they don't have a .py extension even though they are Python scripts.

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

Looking at the logs, I only see .py files are checked, e.g. under flake8
 2 files checked
No problems found

Checked files:

* robot_ws/src/hello_world_robot/setup.py
* robot_ws/src/hello_world_robot/src/hello_world_robot/__init__.py

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

Looking at the logs, I only see .py files are checked, e.g. under flake8
 2 files checked
No problems found

Checked files:

* robot_ws/src/hello_world_robot/setup.py
* robot_ws/src/hello_world_robot/src/hello_world_robot/__init__.py

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

We are also missing files like this, because they don't have a .py extension even though they are Python scripts.

Is having these python files with no extension a best practice? I think adding a .py extension is the solution here.

If only .py files are checked, are we missing most of the files for ROS1? like all the .launch files

We are also missing files like this, because they don't have a .py extension even though they are Python scripts.

Is having these python files with no extension a best practice? I think adding a .py extension is the solution here.

Is having these python files with no extension a best practice? I think adding a .py extension is the solution here.

Yeah, I agree with putting back the .py extension. I'm not aware of excluding the extension being a good practice.

Is having these python files with no extension a best practice? I think adding a .py extension is the solution here.

Yeah, I agree with putting back the .py extension. I'm not aware of excluding the extension being a good practice.


import/export and exchange rxn missing for dihydroorotate

Description of the issue:
The metabolite (S)-dihydroorotate[c] is reported to be exported by cells (see https://www.nature.com/articles/s41467-018-08033-9 ), but there is no export rxn (and I assume there should be an import rxn as well). There is no exch rxn either. I have no idea about GPR though. This is happening in hypoxia.
Expected feature/value/output:
Just add the reactions I'm thinking, i.e.
(S)-dihydroorotate[c] <=> (S)-dihydroorotate[s]
(S)-dihydroorotate[s] <=>
Current feature/value/output:
Reproducing these results:
I hereby confirm that I have:

[ ] Tested my code on my own computer for running the model
[ ] Done this analysis in the main branch of the repository
[ ] Checked that a similar issue does not exist already

Thanks for spotting this. Though I guess now we would abbreviate the compartment as [e] rather than [s].

Looking a bit more into this, the annotation for these reactions doesn't look straightforward.
A name search for (S)-dihydroorotate returned CHEBI:30864, but that is different from CHEBI:17025 which is the annotation of MAM00180c. In fact, the CHEBI name for MAM00180c is (S)-dihydroorotic acid.
Overlooking the potentially misannotated metabolite, some transport reactions for (S)-dihydroorotate seem to exist in MNX.

Yeah, it is with this for many compounds, that you can either talk about the ion or the acid (which includes the H+). It is the same with for example the amino acid glutamate/glutamic acid


Ignore compiled files

When compiling before sourcing the plugin, a zsh-autosuggestions.zsh.zwc is created.
This can be annoying if zsh-autosuggestions is used as a git submodule in a dotfile repo for example, since the repository will then be marked as dirty.
This PR just adds a gitignore that ignores all files ending on .zwc.

If you're compiling multiple zsh plugins it's sensible to add *.zwc to your global gitignore.

Oh wow! I did not know that was possible/did not expect this would solve my problem. Thanks!
I can still see it helping others eventually; what do you think? Otherwise this could be closed.

I'm going to go ahead and close this for now but would consider it in the future if it comes up again.

Sounds reasonable. Thanks!


Support RGB colors with alpha

This is the pattern: rgb(<r> <g> <b> / <a>).
I tried tweaking the tokens comma but it didn't work.

Can you elaborate? It supports RGBA with rgba( r, g, b, a)

@hxf31891 The pattern I mentioned (rgb(<r> <g> <b> / <a>)) will be used for cross-platform communication. The same value will get sent to web, Android and iOS to put together a view based on this.
We chose that pattern for colors because parsing the pieces with the comma pattern (rgba(<r>, <g>, <b>, <a>)) in a gradient string is very hard to maintain and slow, even if not using regexes, which are slow by default.

Merged in v2.0.13


run: Support passing parameters to containerized gadgets

From the design document https://github.com/inspektor-gadget/inspektor-gadget/blob/main/docs/design/002-containerized-gadgets.md#custom-parameters

Some gadgets define parameters in the eBPF code to change its behavior. They are declared with the const volatile keywords. Those constants should be exposed to the user as parameters.
It could be implemented by using the pkg/params package.

FYI, I started looking into it, I already have a dirty POC working -> https://github.com/inspektor-gadget/inspektor-gadget/tree/mauricio/experiments/gadgets-params. Will continue next week.

I am wondering if there is a way to support parameters of type LPM like suggested on https://github.com/inspektor-gadget/inspektor-gadget/pull/2076#discussion_r1335950741
cc @eiffel-fl


Style : New bookings rules

Amendes the booking guidance on /atc/bookings to reflect restrictions on London/Gatwick (EGKK)

:tada: This PR is included in version 6.12.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


refactor!: prefix some commands with onvif service name

BREAKING CHANGE: Changed the name of many device commands to include a prefix of the onvif service they belong to. For example Status becomes PTZStatus, Profiles becomes MediaProfiles, etc.

See new PR https://github.com/edgexfoundry/device-onvif-camera/pull/357


NPM 6 - vulnerability - Memory Exposure

Moderate      │ Memory Exposure                                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ tunnel-agent                                                 │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.6.0                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ node-sass                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ node-sass > request > tunnel-agent                           │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/598

Looks like this was fixed but the author neglected to publish v0.6.1 on NPM.

When will v.0.6.1 be released on NPM?
https://www.npmjs.com/package/tunnel-agent

I'm seeing the fix in my version I got from NPM. Just seems the version number wasn't updated to match. It was a simple one line change, so you should easily be able to see if you have the fix as well. Check line 131 of the index.js file. Link to commit

its been almost a month. Can we please get the NPM package version updated please?

Please publish to NPM!?

@mikeal Please just run npm publish!

Am totally new to the npm audit, I want to know if I can actually continue developing without applying this fix because I have tried updating to the latest 0.6.0 tunnel-agent package but it won't fix the warning.

I am using nvm and since this issue is related to NPM +6, I fixed this with a downgrade to Node 10.2.1 (the last release with npm 5). See the releases.
Running nvm install 10.2.1 has solved this for now.

can someone press npm publish on this? @mikeal @simov

ah, i just saw, its broken downstream; nevermind.

npm run publish 😭

@mikeal I think people are assuming there's a fix for the current NPM memory exposure audit error, why the version bump to 0.6.1 if there's no changes since 0.6.0?

Odd release numbers don't get published, they are only in-tree, this makes it easier to see when people are using git checkouts rather than npm releases.

Oh my. I am still waiting on this to be fixed for npm also, re: https://github.com/request/tunnel-agent/issues/41?_pjax=%23js-repo-pjax-container#issuecomment-447755622
SEMVER WARNING: Recommended action is a potentially breaking change

For everyone coming to this issue after seeing this in their npm audit, the latest version, v0.6.0 has patched this issue. Most likely, the reason you are seeing the advisory is because a dependency nested somewhere in your application is relying on an unpatched version of this package, likely many layers down.
In the example above, node-sass > request > tunnel-agent is the dependency chain that linked to the vulnerable tunnel-agent. You can see this chain under the Path header in npm audit. It's possible that every part of this chain needs to update their dependencies to fix the vulnerability.
Your best bet is to see if you're project's direct dependency, in this case node-sass, has updated their dependencies to a version that is fixed down the line.


SSL is broken

Getting an error creating an app:
$ API_URL=https://f.mydomain.com/ fn apps create myapp
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14dae3d]

goroutine 1 [running]:
main.(*appsCmd).create(0xc42000c090, 0xc4200d7cc0, 0x1, 0x1616500)
	/home/ubuntu/go/src/github.com/iron-io/functions/fn/apps.go:148 +0x6cd
main.(*appsCmd).(main.create)-fm(0xc4200d7cc0, 0x0, 0xc4204b7a30)
	/home/ubuntu/go/src/github.com/iron-io/functions/fn/apps.go:35 +0x34
github.com/iron-io/functions/vendor/github.com/urfave/cli.HandleAction(0x155dca0, 0xc4204b7270, 0xc4200d7cc0, 0x1, 0x0)
	/home/ubuntu/go/src/github.com/iron-io/functions/vendor/github.com/urfave/cli/app.go:485 +0xd4
main.prepareCmdArgsValidation.func1(0xc4200d7cc0, 0x0, 0xc4200d7cc0)
	/home/ubuntu/go/src/github.com/iron-io/functions/fn/main.go:118 +0x357
github.com/iron-io/functions/vendor/github.com/urfave/cli.HandleAction(0x155dca0, 0xc42014c0e0, 0xc4200d7cc0, 0xc420071100, 0x0)
	/home/ubuntu/go/src/github.com/iron-io/functions/vendor/github.com/urfave/cli/app.go:485 +0xd4
github.com/iron-io/functions/vendor/github.com/urfave/cli.Command.Run(0x1617a77, 0x6, 0x0, 0x0, 0xc4204b7330, 0x1, 0x1, 0x16201c7, 0x10, 0x0, ...)
	/home/ubuntu/go/src/github.com/iron-io/functions/vendor/github.com/urfave/cli/command.go:193 +0xb72
github.com/iron-io/functions/vendor/github.com/urfave/cli.(*App).RunAsSubcommand(0xc4202409c0, 0xc4200d77c0, 0x0, 0x0)
	/home/ubuntu/go/src/github.com/iron-io/functions/vendor/github.com/urfave/cli/app.go:374 +0xacc
github.com/iron-io/functions/vendor/github.com/urfave/cli.Command.startApp(0x1615bf3, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x162276d, 0x13, 0x0, ...)
	/home/ubuntu/go/src/github.com/iron-io/functions/vendor/github.com/urfave/cli/command.go:280 +0x824
github.com/iron-io/functions/vendor/github.com/urfave/cli.Command.Run(0x1615bf3, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x162276d, 0x13, 0x0, ...)
	/home/ubuntu/go/src/github.com/iron-io/functions/vendor/github.com/urfave/cli/command.go:79 +0x14c7
github.com/iron-io/functions/vendor/github.com/urfave/cli.(*App).Run(0xc420240820, 0xc420078080, 0x4, 0x4, 0x0, 0x0)
	/home/ubuntu/go/src/github.com/iron-io/functions/vendor/github.com/urfave/cli/app.go:250 +0x7d0
main.main()
	/home/ubuntu/go/src/github.com/iron-io/functions/fn/main.go:126 +0x53

Hello, thanks for all the reports !
This is interesting, it seems fn doesn't get the response it expects.
https://github.com/iron-io/functions/blob/master/fn/apps.go#L148
I'll try to replicate your environment and check for the issue, in the mean time, would it be possible for you to try without https ? (just a hunch, also in light of  #634 )

Without https eh? Okay sure I'll try to figure out how to not make my env secure :D

Okay so your hunch was right :)
$ API_URL=http://node1.mydomain.com:18080/ fn apps create myapp
myapp created

I'll retitle the issue. Let's fi this broken SSL support please :)

NB: This is:
LB (traefik) -> functions service
      HTTPS  ->            whatever

Also the current situation sucks because my environment redirects all http traffic to https and there is no way to turn this off specifically (lack of exclusion rules support) for a particular service/endpoint so functions/faas will not play nicely right now.

thank you, I will look into this.

@prologic sorry for the long delay ! We just released a new version of the fn tool which should solve SSL issues. Please give it a try and let us know if this works for you !
Thanks !

Will do!
On Thu, Sep 28, 2017 at 18:12 Arda Karaduman<EMAIL_ADDRESS>wrote:

@prologic https://github.com/prologic sorry for the long delay ! We
just released a new version of the fn tool which should solve SSL issues.
Please give it a try and let us know if this works for you !
Thanks !
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/iron-io/functions/issues/635#issuecomment-333004396,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABOv-lNVE1jCoKfYXJyqK2xkv1Q8HFZTks5snEP7gaJpZM4O5yyo
.

--
James Mills / prologic
E<EMAIL_ADDRESS>W: prologic.shortcircuit.net.au

closing this for now, please feel free to open again if still not working for you 👍

What was done to fix this?
On Wed, Nov 1, 2017 at 19:08 Arda Karaduman<EMAIL_ADDRESS>wrote:

closing this for now, please feel free to open again if still not working
for you 👍
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/iron-io/functions/issues/635#issuecomment-341297034,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABOv-m6PkGdi-YKSEYze8rjRa1euJtGaks5sySQmgaJpZM4O5yyo
.

--
James Mills / prologic
E<EMAIL_ADDRESS>W: prologic.shortcircuit.net.au

We released a new version on sept 28th.
https://github.com/iron-io/functions/pull/650
You need to update your fn tool and iron/functions container to get the new version.
seems to be working (duplicate issue closed by OP)
https://github.com/iron-io/functions/issues/649

Cool 😀
On Wed, Nov 1, 2017 at 19:17 Arda Karaduman<EMAIL_ADDRESS>wrote:

We released a new version on sept 28th.
#650 https://github.com/iron-io/functions/pull/650
You need to update your fn tool and iron/functions container to get the
new version.
seems to be working (duplicate issue closed by OP)
#649 https://github.com/iron-io/functions/issues/649
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/iron-io/functions/issues/635#issuecomment-341298250,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABOv-ra4KVzz2OSBdIGq1NTCrkIE5ivBks5sySZKgaJpZM4O5yyo
.

--
James Mills / prologic
E<EMAIL_ADDRESS>W: prologic.shortcircuit.net.au


Update hello.c

Changed URL in splash text to point to hub instead of cloud, in reference to issue #45

Green CI! :+1:
Thanks! :heart:


chore: inject otel ctx when dal submits jobs to pinga

This change marks the first transmission of OpenTelemetry trace metadata between 2 service components. In this case, from a NatsProcessor (in the DAL) to a Pinga job. Future work will connect more. Yay!

bors merge

bors retry


[Feature Request]: Async / parallel SemanticSplitterNodeParser

Feature Description
I would like an async version of SemanticSplitterNodeParser, since currently each node's embeddings are generated synchronously, and when a single document has 200-300 embeddings, its unusably slow.
Reason
I tried using an IngestionPipeline, but that doesn't help speed up unfortunately.
Value of Feature
No response

@FireMasterK You mentioned that you did not observe speedup with parallelizing IngestionPipeline on SemanticSplitterNodeParser. Can you share that code snippet as I am not able to get that running as well. I am getting the following error:
Can't pickle local object 'split_by_sentence_tokenizer..split' . I saw the source code and found out that the split method is defined locally inside split_by_sentence_tokenizer and hence cannot be accessed for multiprocessing which is very weird. Can someone suggest a fix/workaround on that?

Hello, has anyone found a solution to the pickle issue?
I am not sure how to remedy the pickle issue.
Any help would be greatly appreciated @dosu-bot .


Various improvements

@systemi-makabe your improvements look great and I want to merge them

closing as superceded by #8