Parquet update

Two changes to Parquet

read_parquet no longer sends around the entire parquet file to every task, reducing communication and serialization costs.  Fixes https://github.com/dask/fastparquet/issues/26
We fuse getitem calls into read_parquet calls to reduce loading data just before we forget it.  Fixes https://github.com/dask/dask/issues/1785

Depends on https://github.com/dask/fastparquet/pull/29

@martindurant any further thoughts here?

I have no further changes

OK, merging this.  Future work might include optimizing getting the index, which currently doesn't optimize down.  Not sure it's worth applying such a check in all cases though.


Add mesh connectivity in atlas

Description
"evaporates the point cloud functionspace"  :cloud_with_rain:

closes #975

Builds a proper atlas functionspace with mesh connectivity.

Geometry constructor has been changed to pull the mesh information from the Fortran side, then pass the constructed functionspace back to Fortran
remove some stuff that is no longer needed (e.g. toFieldSetAD(), passing lonlat from Fortran to C++, the weird hacks that were needed to avoid duplicate halo points)
added an option to save gmsh data when the grid is generated so that it can be viewed with gmsh (see images below, and gridgen.yml)
a bunch of other minor things that were needed because of the first 2 bullet points
references change because the interpolation is slightly different

Testing
note, spack stack 1.5.1 is required for these branches to work.
Here is what the grid looks like with the 5deg grid on 16 PEs

And if you're curious how weird the tripolar fold looks, here is one PE with halos

[x] ctests pass
[ ] test with regional HAT10 on orion
[ ] test with global 1/4

Dependencies
build-group=https://github.com/JCSDA-internal/oops/pull/2465
build-group=https://github.com/JCSDA-internal/saber/pull/729

I was able to build this after the soca-science/#625 on orion that updates spack-stack version. All the ctests pass for soca. I am testing with HAT10 now, will update soon.

@hga007 FYI, in case any of this is useful to you. Yours should be more straightforward since you don't have the northern tripolar fold or cyclic E/W boundaries to deal with. I have code here that does extra checking to see which duplicate halo points need to be removed before being passed to atlas... but you shouldn't have any duplicates to worry about.

@travissluka,  Thank you!  Indeed, it will be easier.  I looked at your code and think it will be easier if I follow your approach.  You did an excellent work implementing this feature. Cheers.

note, tests pass, except for clang. not sure why, I don't have clang.... ignore that one for now.

@fmahebert I could look at your code, but it's probably faster for me to ask you: for the regional FV3, with the "halo" points that aren't owned by anyone because they are at the boundary of the regional domain, do you just assign them to the PE for the patch that is sitting closest to those "boundary halo-ish" points?

@travissluka that's correct. My rationale was that if we later want to apply a proper boundary condition, that would likely be populated into the fv3 halos on that PE anyway.

@kbhargava it should work for hat10 now. I only ran a check on the gridgen, not a full cycle. But I have changed the logic so that the nodes/cells representing the halo outside the domain to the north and east are removed.  (note to self, that in the future we'll want to change that logic when/if we remove the "virtual land" that surrounds our domain)
here is the mesh with HAT10 on 16 PEs

and as a sanity check, a PE in the middle of the domain has halos on all sides:

but the PE on the NE corner does not have halos above and to the right

prep.soca step passes now with the regional case. 🎉

Did you test at "high-res" @travissluka or @kbhargava ? Did you notice a change in the time it takes to run the geometry constructor? Thanks for the pretty figures ...

@guillaumevernieres you doubt my mental ctests??? Fine, let me test with 1/4 deg global.
oh wait...
Node uid:<PHONE_NUMBER>073000000   0 xy(-999.0000000000,-999.0000000000) has already been added as node 0 (-999.0000000000,-999.0000000000)
Node uid:<PHONE_NUMBER>073000000   0 xy(-999.0000000000,-999.0000000000) has already been added as node 0 (-999.0000000000,-999.0000000000)
Node uid:<PHONE_NUMBER>073000000   0 xy(-999.0000000000,-999.0000000000) has already been added as node 0 (-999.0000000000,-999.0000000000)
Node uid:<PHONE_NUMBER>073000000   0 xy(-999.0000000000,-999.0000000000) has already been added as node 0 (-999.0000000000,-999.0000000000)
backtrace [1] stack has 13 addresses
(/home/tsluka/work/spack-stack/envs/skylab.1.5.1_gcc.12.3.0/install/gcc/12.3.0/eckit-1.24.4-tk6ui5d/lib/libeckit.so+eckit::BackTrace::dump[abi:cxx11]())0x1db
(/home/tsluka/work/spack-stack/envs/skylab.1.5.1_gcc.12.3.0/install/gcc/12.3.0/eckit-1.24.4-tk6ui5d/lib/libeckit.so+eckit::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, eckit::CodeLocation const&, bool))0xb3
(/home/tsluka/work/spack-stack/envs/skylab.1.5.1_gcc.12.3.0/install/gcc/12.3.0/ecmwf-atlas-0.35.0-tez5dnr/lib/libatlas.so.0.35+atlas::throw_Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, eckit::CodeLocation const&))0x29
(/home/tsluka/work/spack-stack/envs/skylab.1.5.1_gcc.12.3.0/install/gcc/12.3.0/ecmwf-atlas-0.35.0-tez5dnr/lib/libatlas.so.0.35+atlas::mesh::actions::build_lookup_uid2node(atlas::Mesh&, std::unordered_map<long, int, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, int> > >&))0xdca
(/home/tsluka/work/spack-stack/envs/skylab.1.5.1_gcc.12.3.0/install/gcc/12.3.0/ecmwf-atlas-0.35.0-tez5dnr/lib/libatlas.so.0.35+atlas::mesh::actions::increase_halo_interior(atlas::mesh::actions::BuildHaloHelper&))0x3d8
(/home/tsluka/work/spack-stack/envs/skylab.1.5.1_gcc.12.3.0/install/gcc/12.3.0/ecmwf-atlas-0.35.0-tez5dnr/lib/libatlas.so.0.35+atlas::mesh::actions::BuildHalo::operator()(int))0x53b
(/home/tsluka/work/soca/build_gcc11/bin/../lib/libsoca.so+soca::Geometry::Geometry(eckit::Configuration const&, eckit::mpi::Comm const&, bool))0xfbe
(./soca_gridgen.x+)0x268c
(/home/tsluka/work/soca/build_gcc11/bin/../lib/liboops.so+oops::Run::execute(oops::Application const&, eckit::mpi::Comm const&))0x2c2
(./soca_gridgen.x+main)0x5a
(/lib/x86_64-linux-gnu/libc.so.6+)0x280d0
(/lib/x86_64-linux-gnu/libc.so.6+__libc_start_main)0x89
(./soca_gridgen.x+)0x2365

@guillaumevernieres @kbhargava never mind, false alarm!
I was trying to test 1/4 deg but my MOM_input file did not have TRIPOLAR_N=True so bad things were happening. We'll have to update the static files for skylab because I don't think MOM_input has that set.
I only tested the grid generation, I did not test a full cycle (I leave that as an exercise for the user!). Here is 1/4 deg with 32 PEs

and the last PE along the tripolar fold showing halos

and @guillaumevernieres there is no noticeable difference in run time, creating the atlas mesh is fast.

The OOPS PR has been merged; this PR can be merged as well.
(Unless if the clang CI needs more attention?)

The OOPS PR has been merged; this PR can be merged as well.
(Unless if the clang CI needs more attention?)

@fmahebert  the clang CI was apparently breaking with NaNs before this PR... i just never noticed it. Deal with that later.


pm_list: Use "Muted user" placeholder for muted usernames.

This also handles a few other places missed earlier like
narrow headings, beacuse they use the same function.
We already rerender the PM list for events, so there's no
need to do anything special when someone is muted/unmuted.

I also grepped for patterns like map((user_id) => get_full_name(user_id) and there aren't any remaining after this.

Merged, thanks @abhijeetbodas2001!


Pipeline Graph from Interactive Beam -- made faster

Optimization
Changed filtering top level PTransform by string manipulation to
searching for them directly through looking into subtransforms of root
Ptransforms. (see _top_level_transforms()) Makes PipelineGraph faster.

Generalization

Moved display_graph() method from InteractivePipelineGraph to PipelineGraph
PipelineGraph now takes either pipeline obj or proto

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.
 Are you looking for a review of this?

Yes! It'd be great if you could review it. Thanks!

Sure. L take a look tomorrow.
On Thu, Aug 16, 2018, 5:44 PM Qinye Sindy Li<EMAIL_ADDRESS>wrote:

Yes! It'd be great if you could review it. Thanks!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/pull/6227#issuecomment-413725013, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABPc7EcRvIEOZGfF5KrzzhWHasdd7s8wks5uRhHygaJpZM4V9V-f
.

--
Got feedback? go/pabloem-feedback

I'll merge after precommits


False Positive on Quarkus

Quarkus (registered in NVD as quarkus:quarkus) 1.12.0.Final depends on:
<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-security</artifactId>
   <version>1.1.3.Final</version>
</dependency>

(Note we also have io.quarkus/quarkus-http and io.quarkus/gizmo Maven dependencies - these are registered in NVD as quarkus:quarkus-http and quarkus:gizmo)
Before quarkus-security has been registered in NVD, we were seeing quarkus-security:1.1.3.Final matching CVE-2017-18640 - which was opened before Quarkus projtect has been created.
After registering it as quarkus:quarkus-security in NVD we have started seeing it matching the main quarkus:quarkus CPE:
https://github.com/quarkusio/quarkus/issues/2611#issuecomment-804060800
Please see  https://github.com/quarkusio/quarkus/issues/2611 for more information.

Hi @sberyozkin :wave:
I looked at your request, and the links you provided. If I understand correctly, you want all quarkus packages with the following group ids:

io.quarkus.security
io.quarkus.http
io.quarkus.gizmo

not being attached to CPE quarkus:quarkus because they have their own. Am I right?
I have prepared a commit following this rule. I will wait for your confirmation before opening a PR to be sure I'm not going to submit an undesired configuration.

Hi @nhumblot and @jeremylong
Sorry I have missed your ping - yes, exactly, these 3 group IDs have their own CPE qualifiers.
Thanks for looking into it and adding this rule :-)
We'll wait for the release of 6.1.7
Cheers

Sorry, see you are planning it for 6.2.0 - that is fine, thanks


Improve performance of allocating a List with a known length with null safety

A common pattern when allocating a large list where the length is known ahead of time is to use the default List constructor. This avoids needing to repeatedly grow the internal buffer, though there might have been other optimizations that applied too.
var result = List<int>(1000);
for (var i = 0; i < 1000; i++) {
  result[i] = i;
}

In a null-safe world, this API has been removed since it required filling the list with nulls . There are several alternative APIs that could be used.

List.add

var result = <int>[];
for (var i = 0; i < 1000; i++) {
  result.add(i);
}

Collection for

var result = <int>[
  for (var i = 0; i < 1000; i++) i
];

List.generate

int fill(int x) => x;
var result = List<int>.generate(1000, fill);

Preallocating the list is between 2 - 4x faster than any of these alternative methods unfortunately. This feels bad because it will only impact users that already needed to look for performance. Whether or not that performance is needed is hard to say, but it would be nice to have a good replacement for this optimization.
Looking at the kernel output for the Collection for, it essentially de-sugars into repeated calls to List.add:
  static method generateWithElement() → core::List<core::int*>* {
    return block {
      final core::List<core::int*>* #t1446 = <core::int*>[];
      for (core::int* i = 0; i.{core::num::<}(1000); i = i.{core::num::+}(1))
        [@vm.call-site-attributes.metadata=receiverType:InterfaceType(List<int*>*)] #t1446.{core::List::add}(i);
    } =>#t1446;
  }

And while List.generate) does preallocate the right size, it also requires a closure and seems to be running marginally slower than List.add in AOT
It seems like the easiest way (though I really have no idea) to get some performance back here would be to set an initial capacity in the collection for, if a some minimal length is known. This could apply if a collection for is used with a loop or List with no nested collection ifs.
The performance measurement was done with this benchmark running a flutter release build on android-arm64. Note: its not a very good benchmark, the best way to work around that is to comment out all but one of the methods for testing.
FYI @leafpetersen @yjbanov

I'm curious if List.filled() happens to be any better than List.generate in this regard. The core/lib code for List shows it as an external factory, rather than for-loop containing function that List.generate() is, so that might be enough difference to return the performance you're wanting?

I did not measure List.filled since it does not work for types without a meaningful zero value, nor does it allow the values themselves to vary.
If you click on the link above for List.generate and scroll up, you can see the implementation of List.filled. this works by preallocating the list length and then filling in each element, the sane way you would write pre null-safety

I think we should force inlining of List.generate(n, () => ...) and the closure invocation itself - the only problem would be potential impact on the code size. Though closure allocation should disappear (and with it the closure metadata and body) so maybe it will be okay.
I think we should tweak inlining heuristics in general to force inlining of local closures with a single call site.
@alexmarkov Alex would you like to look at this, given that you dealt with other NNDB related performance issues?
@sstrickl FYI (related to inlining heuristics).

We can separately look at optimizing collection-for syntax to preallocate resulting array of the right size - should not be that hard to do that even on already desugared code, e.g.
var list = [];
for (var i = 0; i < n; i++) {
  list.add(i);
}

If we know that list does not escape in the loop and we know the loop boundary then we can preallocate list to have capacity of n.

After https://dart-review.googlesource.com/c/sdk/+/151082 and https://dart-review.googlesource.com/c/sdk/+/151466 the code
  var result = List<int>.generate(1000, (int x) => x, growable: false);

is now on par with
  var result = List<int>(1000);
  for (var i = 0; i < 1000; i++) {
    result[i] = i;
  }


fix typo: merg -> merge

fix typo

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.wangzhijun seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.wangzhijun seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


chore: add redis to docker compose

Current coverage is 87.70% (diff: 100%)

Merging #38 into master will not change coverage

@@             master        #38   diff @@
==========================================
  Files             2          2
  Lines           122        122
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            107        107
  Misses           13         13
  Partials          2          2

Powered by Codecov. Last update ae06ecc...3e9e796


Add support for Float

Additions

Added #Floatto the JSON variant
Added a parser to convert text to Float
Added tests for show() and parse()

Parsed fractions do not match the fractions defined in Motoko but are nearly equal.
1.234e-4 != 1.234e-4

Also, I tried to implement the number fraction exponent in this https://github.com/aviate-labs/json.mo/issues/3, but I think there will be some confusion because motoko only recognizes the format as a Float.


Transitive dependencies from generate-include-scripts fail to load in Fsi

Description
In Linux (using Mono <IP_ADDRESS>) the resolution behavior doesn't take into account transitive assemblies of any include script generated by Paket in your main script. Transitive dependencies from I can see are affected by the issue and FSI fails to evaluate/load the transitive assembly.
Repro steps
Please provide the steps required to reproduce the problem

paket generate-include-scripts to generate the F# scripts.
Reference the appropriate include script.
Try using a type from an generated include script that has transitive dependencies on another Nuget package. (e.g a type inported from something like #load "include.main.fsx").

Expected behavior
F# script to just work with the correct dependencies.
Actual behavior
/path/to/current/dir/stdin(0,1): error FS0078: Unable to find the file 'TransitiveDep.dll' in any of
/usr/lib/mono/4.5
Known workarounds
For every assembly load (#r) generated by Paket do it instead via two steps (e.g if assembly is ./paket-files/net46/Newtonsoft.Json.dll):

Using the #I to register the directory ( #I /paket-files/net46/)
Loading the #r with or without the directory. (#r Newtonsoft.Json.dll)

If all assembly loads on all generated scripts are done this way it all works as expected.
Believe this is related to: https://github.com/fsharp/fsharp/issues/251.

/cc @smoothdeveloper


Unable to call a platform channel method from another isolate

Unable to call a platform channel method from another isolate
can't use platform channels on a separate isolates
how can i do it. thank you.

@chongbo2013
it's not a bug, isolates don't share memory

Closing this as duplicate of https://github.com/flutter/flutter/issues/13937


Add watch function to KV API

The contents function is basically a watch function that times out after 100ms. I'd like the ability to request a long-lived consumer of key-value pairs so that every time a new key is added (or deleted), my consumer gets notified of this.
My initial attempt was to write a watch function and then modify the contents function to basically be a specialized consumer of watch. However, I quickly ended up unsure on how to proceed.
What I was trying to implement:
def watch(conn, bucket_name, module, keys \\ "*")

Where module was a module that uses GnatServer.. but what I really want is to receive a message like this:
{:key_added, key, value}
{:key_removed, key, value}
{:key_updated, key, value}

So that felt more like something that made more sense with a Gnat.sub .. but again I wasn't sure how to translate the stream into these three pieces because the documentation on the headers used for KV is sparse to say the least.

I'm more than happy to contribute this, I think I just want some directional advice on what you think is the best approach here.

here's some as-yet-untested code (brain-compiled) that describes what I was thinking:
def watch(conn, bucket_name, handler) do
    stream = stream_name(bucket_name)
    inbox = Util.reply_inbox()
    consumer_name = "all_key_values_watcher_#{Util.nuid()}"

    {:ok, int_handler} = Task.start(fn ->
      receive do
        {:msg, %{topic: key, body: body, headers: headers}} ->
          key = subject_to_key(key, bucket_name)

          if {"kv-operation", "DEL"} in headers do
            handler.(:key_deleted, key, body)
          else if {"kv-operation", "PUT"} in headers  # ??? PUT ?
            handler.(:key_added, key, body)
          else
            handler.(:key_updated, key, body)
          end
      end
    end)

    with {:ok, sub} <- Gnat.sub(conn, int_handler, inbox),
         {:ok, _consumer} <-
           Consumer.create(conn, %Consumer{
             durable_name: consumer_name,
             deliver_subject: inbox,
             stream_name: stream,
             ack_policy: :none,
             max_ack_pending: -1,
             max_deliver: 1
           }) do

        {:ok, sub}
    end
  end

And then it would be up to the caller of watch to decide when to unsubscribe and remove the consumer (probably through unwatch that takes the bucket name and the sub pid?)

Released as 0.0.6


Bug or Feature Removed: Unable to swap ammunition type in new system 4.0.2

In the old system ammo could be chosen among the ones available so a player could choose normal arrow, +1, etc. Now there is no option to select the ammunition used on weapons with ammunition.

This is done during the attack roll now.


Refactor the page interface

https://saber.land/docs/page-interface.html
We should not reuse the page.attributes object, i.e. setting a new page.permalink property instead of overriding page.attributes.permalink
Currently the behavior is inconsistent, sometimes we use page.attributes.permalink the other times we use page.excerpt 😅
Basically we will keep page.attributes which is set by front matter or export const attributes = {}, instead of modifying page.attributes we should infer page from page.attributes and other information, e.g. page.slug is inferred from page.attributes.slug and the filename.
This will introduce breaking changes.

IssueHunt Summary

Backers (Total: $100.00)

 issuehunt ($100.00)

Become a backer now!
Or submit a pull request to get the deposits!
Tips

Checkout the Issuehunt explorer to discover more funded issues.
Need some help from other developers? Add your repositories on IssueHunt to raise funds.

IssueHunt has been backed by the following sponsors. Become a sponsor

Hi, can I work on this? I’ve used Saber for a few weeks and really like it.

Sure, if having any questions, feel free to ask.
saltysugar<EMAIL_ADDRESS>于 2019年7月21日周日 上午12:18写道：

Hi, can I work on this? I’ve used Saber for a few weeks and really like it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/saberland/saber/issues/332?email_source=notifications&email_token=ACDAWSHNL77UAB7WIGQV6Z3QAM3GJA5CNFSM4IDV4242YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NRM6Y#issuecomment-513480315,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACDAWSFY53ZZCM4UL4FWKUTQAM3GJANCNFSM4IDV424Q
.

Hmm I'm thinking about making this more similar to Jekyll:

https://jekyllrb.com/docs/front-matter/
https://jekyllrb.com/docs/variables/

The page variable in Jekyll is simply page specific information + front matter, i.e. custom variables set via the front matter will be available on page.


Could not interpret loss function "NDCG Lambda Weight V2"

Click to expand!
Issue Type
Feature Request
Have you reproduced the bug with TF nightly?
Yes
Source
source
Tensorflow Version
2.12
Custom Code
Yes
OS Platform and Distribution
colab
Mobile device
colab
Python version
3.9
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
A erro happened!
Could not interpret loss function NDCG Lambda Weight V2
ValueError: Could not interpret loss function identifier: <tensorflow_ranking.python.keras.losses.NDCGLambdaWeightV2 object at 0x7fa0e84ee6e0>
Standalone code to reproduce the issue
model.compile(optimizer=tf.keras.optimizers.AdamW(learning_rate, beta_1=0.9, beta_2=0.98,epsilon=1e-9)
              loss=tfr.keras.losses.NDCGLambdaWeightV2(topn=60),
              metrics=[tfr.keras.metrics.NDCGMetric(topn=60), tfr.keras.metrics.OPAMetric()])
model.fit(train_dataset, epochs=20)

Relevant log output
No response

may should  be this one
loss = tfr.keras.losses.ApproxNDCGLoss(lambda_weight=tfr.keras.losses.NDCGLambdaWeightV2(topn=60))

@mega-TWZ Thank you for reporting an issue. Could you provide more details on this issue for which you are requesting the feature. Please share the complete standalone code to replicate the issue.
Thank you!

tfr-->keras-->losses
The intention here is a bit vague, we should better distinguish between 'lambda_weight' and 'losses', rather than placing them in the same directory
截图](https://github.com/tensorflow/tensorflow/assets/77107277/5548c89b-9bb9-4a14-ba93-0964a03730c6)

@mega-TWZ感谢您报告问题。您能否提供有关您请求该功能的这个问题的更多详细信息。请共享完整的独立代码以重现问题题。 谢谢你！

tfr --> keras --> losses
The intention here is a bit vague, we should better distinguish between 'lambda_weight' and 'losses', rather than placing them in the same directory
a picture of tfr.keras.losses (https://github.com/tensorflow/tensorflow/assets/77107277/bc2b6ba7-5868-49cf-899c-d4bc92a610e0)

@mega-TWZ Thank you for reporting an issue. Could you provide more details on this issue for which you are requesting the feature. Please share the complete standalone code to replicate the issue. Thank you!

you can see my first comment, I have re-edit it

@mega-TWZ Thank you for your response!
This issue seems to be Keras feature.  Please post this issue on keras-team/keras repo. as Keras development is fully moving to github.com/keras-team/keras. All issues and PRs related to keras will be addressed in that repo.
To know more see this TF forum discussion ;
https://discuss.tensorflow.org/t/keras-project-moved-to-new-repository-in-https-github-com-keras-team-keras/1999
Thank you!


order of execution of functions in cond statement is inconsistent

Inconsistency and/or feature request.
Tensorflows control flow method
tf.cond(condition, fn1, fn2)
executes both functions fn1 and fn2 and returns only one depending on the evaluation of condition. In addition, the order in which the two functions are evaluated varies or is undefined as the following code shows:
var1 = tf.Variable(tf.zeros(1), trainable=False, name='var1')
var2 = tf.Variable(tf.zeros(1), trainable=False, name='var2')
update_var1 = tf.assign(var1,var1 +1)
update_var2 = tf.assign(var2,var2 +1)
training = tf.placeholder(tf.bool)
def f1():
  with tf.control_dependencies([update_var1]):
    return var1 + 10*var2
def f2():
  with tf.control_dependencies([update_var2]):
    return var1 + 10*var2

final = tf.cond(training, f1, f2)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
for i in range(10):
  print(sess.run(final, feed_dict={training: False}))

The output of this varies from run to run, but a typical example is:
[ 10.]
[ 22.]
[ 33.]
[ 44.]
[ 55.]
[ 66.]
[ 77.]
[ 87.]
[ 98.]
[ 110.]

For the lines with 10, 87 and 98 the order of execution apparently was:

update var1
evaluate and return var1 * 10*var2
update var2

while for the lines with 22, 33, 44, 55, 66, 77 and 110 the order of execution must have been:

update var1
update var2
evaluate and return var1 * 10*var2

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
See code in description
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Linux
TensorFlow installed from (source or binary):
from binary
TensorFlow version (use command below):
('v1.2.0-rc2-21-g12f033d', '1.2.0')
Python version:
Python 2.7.13 |Anaconda 4.3.1 (64-bit)
Bazel version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:
Exact command to reproduce:

If you create the tf.assign() ops outside the branch functions, this is working as intended. I answered the question on Stack Overflow with an explanation of how to make this consistent.


PHP 8.1 support

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Is your feature request related to a problem? Please describe.
Although 8.0 was our goal for the year if we can move to 8.1 that would be even better.
Describe the solution you like
Phalcon 5 needs to support it first. Likely to be carried out after 22.10 business release.

Check https://www.php.net/manual/en/migration81.php but otherwise builds and runs fine for 23.1 beta already.


Add new block

priority:medium status:in-progress type:feature

@lpareja99

Problem with wallets: change wallArr to wallets. AKA passes it through the methods and does not call it from import to be able to use it later to create a block.
Make sure the whole app is still working after that.

DONE

DONE


Added Support for Syncronous Testing of Promise Presence

While the resolution of a promise is clearly an asynchronous thing, testing for the presence of a promise is not, so it should be possible to test whether or not a given value is a promise using validate() or validate.single().
This very small commit makes that possible by adding an option allowPromises to validate().
The following console output from node run from the project root folder shows the proposed change in action:
cc-dsk-2ss:validate.js bart$ node
> let validate = require('./');
undefined
> // create a custom promise validator
undefined
> validate.validators.promise = function(val, opts){
...     // implicitly pass empty values
...     if(!validate.isDefined(val)) return undefined;
...
...     // test the passing case
...     if(validate.isPromise(val)){
.....         return undefined;
.....     }
...
...     // if we got here, we have an error, so return an error message
...     return opts.message || this.message || 'is not a promise';
... };
[Function]
> // create a promise
undefined
> var p = Promise.resolve('thingys');
undefined
>
> // should throw an error
undefined
> validate.single(p, {presence: true, promise: true});
Error: Use validate.async if you want support for promises
    at validate (/Users/bart/Documents/Temp/fromGitHub/validate.js/validate.js:33:17)
    at Function.single (/Users/bart/Documents/Temp/fromGitHub/validate.js/validate.js:202:14)
    at repl:1:10
    at ContextifyScript.Script.runInThisContext (vm.js:44:33)
    at REPLServer.defaultEval (repl.js:239:29)
    at bound (domain.js:301:14)
    at REPLServer.runBound [as eval] (domain.js:314:12)
    at REPLServer.onLine (repl.js:433:10)
    at emitOne (events.js:120:20)
    at REPLServer.emit (events.js:210:7)
> // should fail validation
undefined
> validate.single('stuff', {presence: true, promise: true}, {allowPromises: true});
[ 'is not a promise' ]
> // should pass validation
undefined
> validate.single(p, {presence: true, promise: true}, {allowPromises: true});
undefined
>

I could also add the promise validator as a separate pull request?

Coverage remained the same at 100.0% when pulling ddc1a85b0c7509f0add05b40d1beeedaed9ccb03 on bbusschots-mu:master into 07f64ec81e8dd71d1553c20d750fbb3b7ed40018 on ansman:master.

This has actually been a long standing bug.

Closed via bfa56eb

A fix has been released in 0.12.0


OSError: AF_UNIX path too long

Describe the bug
Since the release of behaveX 3.2.0 some of our builds have started to fail with 'AF_UNIX path too long' error.
(By saying 'some builds', I mean that we noticed that for the same repo a build for master is still OK, but a build for a branch has started to fail, even though is was OK yesterday.)
By pinning behaveX version to 3.0.0 we have achieved that our build is successful in all repos and branches.
To Reproduce
Steps to reproduce the behavior:
15:32:24  + python3 -m behavex --parallel-processes 8 --parallel-scheme scenario --logging-level ERROR '--tags=~Isolated'
15:32:25  |--------------------| ------------------------------------------------------------|
15:32:25  |ENV. VARIABLE       | VALUE                                                       |
15:32:25  |--------------------| ------------------------------------------------------------|
15:32:25  |HOME                | /home/ec2-user                                              |
15:32:25  |CONFIG              | /home/ec2-user/.local/lib/python3.8/site-packages/behavex/conf_behavex.cfg|
15:32:25  |OUTPUT              | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/output|
15:32:25  |TAGS                | ~Isolated                                                   |
15:32:25  |PARALLEL_SCHEME     | scenario                                                    |
15:32:25  |PARALLEL_PROCESSES  | 8                                                           |
15:32:25  |FEATURES_PATH       | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/features|
15:32:25  |TEMP                | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/output/temp|
15:32:25  |LOGS                | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/output/outputs/logs|
15:32:25  |LOGGING_LEVEL       | ERROR                                                       |
15:32:25  |--------------------| ------------------------------------------------------------|
15:32:25  Process SyncManager-1:
15:32:25  Traceback (most recent call last):
15:32:25    File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
15:32:25      self.run()
15:32:25    File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
15:32:25      self._target(*self._args, **self._kwargs)
15:32:25    File "/usr/lib64/python3.8/multiprocessing/managers.py", line 608, in _run_server
15:32:25      server = cls._Server(registry, address, authkey, serializer)
15:32:25    File "/usr/lib64/python3.8/multiprocessing/managers.py", line 154, in init
15:32:25      self.listener = Listener(address=address, backlog=16)
15:32:25    File "/usr/lib64/python3.8/multiprocessing/connection.py", line 448, in init
15:32:25      self._listener = SocketListener(address, family, backlog)
15:32:25    File "/usr/lib64/python3.8/multiprocessing/connection.py", line 591, in init
15:32:25      self._socket.bind(address)
15:32:25  OSError: AF_UNIX path too long
Expected behavior
(captured after pinning behaveX to 3.0.0)
16:04:03  + python3 -m behavex --parallel-processes 8 --parallel-scheme scenario --logging-level ERROR '--tags=~Isolated'
16:04:03  |--------------------| ------------------------------------------------------------|
16:04:03  |ENV. VARIABLE       | VALUE                                                       |
16:04:03  |--------------------| ------------------------------------------------------------|
16:04:03  |HOME                | /home/ec2-user                                              |
16:04:03  |CONFIG              | /home/ec2-user/.local/lib/python3.8/site-packages/behavex/conf_behavex.cfg|
16:04:03  |OUTPUT              | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/output|
16:04:03  |TAGS                | ~Isolated                                                   |
16:04:03  |PARALLEL_SCHEME     | scenario                                                    |
16:04:03  |PARALLEL_PROCESSES  | 8                                                           |
16:04:03  |FEATURES_PATH       | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/features|
16:04:03  |TEMP                | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/output/temp|
16:04:03  |LOGS                | /mnt/jenkins/workspace/orders_SP-896-Add-Non-Sales-Type/src/test/output/outputs/logs|
16:04:03  |LOGGING_LEVEL       | ERROR                                                       |
16:04:03  |--------------------| ------------------------------------------------------------|
16:04:27  HTTP API_ID: ************
16:04:27  HTTP URL: https://**************************
16:04:27  HTTP Status: 200
16:04:27  HTTP Body: "{ "status": "success", "data": null }"
16:04:27  HTTP API_ID: *********
16:04:27  HTTP URL: https://******************************
16:04:27  HTTP PARAM: "2023-11-23 15:04:06:*********"
16:04:27  HTTP Status: 200
16:04:27  HTTP Body: "{ "status": "success", "data": { ******
AND SO ON
Running on
Aws Linux image packed by hashicorp.com (latest)
installed: git, gcc, java-1.8.0-openjdk, mysql, mysql-connector-java, mysqldef_linux_amd64.tar.gz, xorg-x11-server-Xvfb, jfrog.io, Node.sh

Hi @Roman-Churakov, maybe the issue is related to a small change in the way we are using the multiprocessing library to start implementing a lock in a shared area.
We will analyze this. In the meantime, I have seen a workaround can be to export the environment variable $TMPDIR to a shorter path.
Regards,

@Roman-Churakov this was a worthwhile workaround for us:
TMPDIR=/tmp/ behavex -t=@tag --parallel-processes=10 --parallel-scheme=scenario

(instead of the usual behavex -t=@tag --parallel-processes=10 --parallel-scheme=scenario)

above command is not working  CategoryInfo          : ObjectNotFound: (TMPDIR=/tmp/:String) [], CommandNotFoundException


Deadlock when registering process

First off, great job on the library! I can't believe how easy it was to incorporate it into my project. There is one issue that I found though. When spawning lots of processes very rapidly across multiple nodes, I often run into a deadlock. I've tracked the issue to this line
https://github.com/bitwalker/swarm/blob/ec943bde2e5741e5a475b8bbe42f433645da9f06/lib/swarm/tracker.ex#L123
It seems both nodes try to call each other's Swarm.Tracker at the same time and time out when none of them can reply.

I made a quick fix by deferring the GenServer.call
spawn_link(fn ->
  res = GenServer.call({__MODULE__, node}, {:register, name, mfa, groups}, 5_000)
  GenServer.reply(from, res)
end)
{:noreply, state}

It's probably not the way to go, I just wanted to see if that solves the issue. Is there any particular reason that registration has to go through the Swarm.Tracker GenServer?

Ah good catch, that is indeed the problem. Registrations go through the tracker process in order to enforce serialized writes to the registry, but the more I think about it, the less I think it may be necessary. I'm at ElixirConf right now, so I have limited time to tackle this, but I'll get a fix released as soon as possible.

Thanks! Though, don't you already enforce serialized writes through the Swarm.ETS GenServer?

@szlend I'm getting ready to merge a set of changes which rework a lot of the internals. Would you be willing to give the pubsub branch a shot? You should be able to just swap out your dependency with {:swarm, github: "bitwalker/swarm", branch: "pubsub"} to try it out.
Your original issue has been addressed there, I just want to be sure the new branch works as expected for more than just myself before I merge into master and release it.

Thanks for tackling this one!  Here's some quick feedback, I'll try and do some more proper testing this weekend.

I needed to replace the process name with a binary (it was a tuple before). I'm guessing this is due to Phoenix PubSub. It's not a problem for me, just something I noticed.
Every so often I get this error when running my application. It seems to happen completely random.
The on_load function for module crypto returned {:error, {:load, 'Library load-call unsuccessful.'}}. This one is really weird cause I can load and use :cryptojust fine on it's own in iex.
I'm not so sure that automatic conflict resolution is that great for my use case. Or maybe I just don't understand how I'm supposed to handle it properly. How can I ensure I don't lose any process state in case of a conflict?
I did eventually run into a timeout when registering the process. But it's possible that I completely flooded the scheduler. I'll try and do some more proper tests later.

@szlend Thanks for the feedback! A couple notes:

A tuple should be supported, I was testing with atoms, but I must've missed something, I'll get that fixed right away.
I don't know that this is related to Swarm specifically. :ssl is in the applications list of Swarm, which depends on :crypto, but the error is likely due to something in your environment I think. That said, I don't need to have :ssl in Swarm, so I can remove it.
There is documentation on how to handle the {:swarm, :begin_handoff} call, if you don't want Swarm to move processes around you can return :ignore, but they will be restarted if the node they live on dies or becomes unavailable. In the case of a network partition, the most recent registration wins (since it would be the one which places the process on the right node), however I can probably add an option to prevent doing so in cases where you don't want Swarm to do anything with those processes other than the initial distribution.
Could you give me any numbers on how many processes you were registering and how fast? Were you dropping/adding nodes during this as well? That will help me figure out where the bottleneck may be. I haven't done heavy stress testing on it yet myself, though I will be very soon, since I was trying to make sure it worked correctly before making it scale, but there are a few places I can think of which may be causing bottlenecks.

Thanks again for the help! I'll be pushing those changes I mentioned today :)

Great!

It appears to be a bug with Erlang/OpenSSL. If you start :crypto after:observer, it fails to load OpenSSL. I reported the issue on the Erlang bug tracker.

What happens if the old process receives a message in the middle of a handoff? I'm guessing this won't be forwarded to the new process right? Do I have to handle such edge cases by myself?

Thousands of processes and probably hundreds of processes per second. I was writing a distributed minesweeper grid, with each 100x100 section being it's own process. I lazily spawn more sections as you reveal tiles on the edge of each section. Then to stress test the communication between the sections, I set the section size to 2x2 and ran the reveal function on an almost empty grid, so it expands into infinity. I didn't add or remove any nodes during the stress test.

Also for fun, here's the comparison of scheduler utilization between versions.
0.4 https://i.imgur.com/x252OmK.png
0.5 https://i.imgur.com/8cdNCb7.png

That's very interesting, looks like average utilization is probably about the same, but the new version is much more bursty. Also interesting that the memory and IO usage is much lower in the new version.

Oh that's very interesting, I haven't hit that one before, well done figuring it out!

That is up to the consumer of the API for now, in my case it doesn't matter, and the window for problems to arise is very short, as immediately after getting the response from handle_call, {:swarm, :die} is sent to the process. There is probably a way to do the response and shutdown in one shot, I know I can think of a way off the top of my head (using the approach in your workaround, reply asynchronously and return {:stop, :shutdown, nil} to kill the process and prevent handling any further messages).

Ok that's good to know, I'll do some testing and see if I can reproduce, probably will be sometime tomorrow.

Thanks again for the help!

I also noticed that in my case it's creating a ton of duplicate processes/registrations that it later has to resolve (kill). Even with a single node. I don't think Phoenix Presence is that great for registering processes on the same node, because it introduces an unnecessary race condition between spawning and registration. Would it be possible to write local registrations into ETS immediately? That way you don't have any conflicts with spawning, and you can use Presence only to sync registrations and handle node changes. Or am I missing some potential issues with that?

If you're interested in trying my latest shot at rewriting the internals, it's on the rewrite branch. It's missing process groups at the moment, though that's by far the easiest thing to add, and there are a few edge cases around conflict resolution I have on my todo list, but haven't yet implemented fixes for, but in the general case it should work much better, and I'd be interested to see how it works with other's use cases. Any feedback is helpful :)

Awesome, I'll try and have a look this weekend :)

Just FYI, I've changed the branch to develop, so you don't wonder why rewrite is missing :)

I just wrote a small benchmark for swarm. It's similar to what I do in my project, but a lot worse. Basically what it does is it starts a process called n and then it tires to ping m (n-1, n+1, n-2, n+2, ...) amount of processes around it. And if the process doesn't exist, it lazily spawns it. The next process then repeats that until it reaches a certain depth.
It seems to work fine with just one node, but as soon as I add another it just keeps spawning processes until it runs out of resources. Though maybe it's because I haven't handled conflict resolution. It's weird though cause it doesn't print a single line when distributed.
I don't know if this is a very realistic case that you want to handle, but on my project I currently use synand hash_ring and it seems to handle that kind of workload fine.
https://github.com/szlend/swarm_test
I hope this helps you in any way :)

@szlend Thanks :). I'm still tweaking some things, so it looks like your test found a bug I need to address.
I actually tried an implementation using the same internals as syn, but the problem is that mnesia cannot handle dynamic node membership, so unless you have a fixed set of nodes, it's not an option (which is my situation).
I'll keep updating this thread as I get closer to a release!

Oh, one thing you may want to do in your project is use my hash ring implementation :)
It's about 10x faster than hash_ring, and is based on the same algorithm. The only difference is that you need to maintain the hash ring state in your own process (I don't wrap it in a GenServer because going through a process is a bottleneck I wished to avoid).

Oh cool, I'm surprised it runs so fast considering hash_ring was written in C! But currently my code doesn't work with adding/removing nodes anyway so maybe using a hash ring is kind of unnecessary anyway. But I'll definitely give it a shot once I start supporting that. You should really extract it into it's own library :P

@szlend Yeah I was very surprised as well, either the data structure I'm using for the ring (gb_tree) is faster for the kinds of operations being performed than the one in hash_ring, or there is some other bottleneck.
I'm planning on extracting it to a separate library for sure, just not sure what to call it haha.

@szlend I think I understand now why your test app is failing with multiple nodes: Swarm sends registrations to the node which is responsible for starting the process, Swarm then broadcasts the registration for replication, but the original node which received the request does not track the registration itself, it replies to the caller but waits for the broadcast to track the registration to avoid conflicts. I'm going to add an overload of register_name which takes options, one of which will be to await the broadcast to ensure the registration is replicated before replying to the caller, this would resolve the issue your app has. I'll follow up here when I've got that done  along with some testing.

@szlend I just pushed changes to develop which fix the issues you discovered with your test app based on my test runs. Could you give it a shot and let me know what feedback you have?

I'm going to close this issue now that it's grown rather large and a new release has been cut. Can you open a new one if you discover any issues with the latest release?

Sure, thanks! I'll test it out as soon as I find some free time :)


Fat/uber Jar doesn't work because of the multi-release feature

I'm trying to build a jar file for my Java program that depends on Skija with the help of the Gradle Shadow plugin. It builds fine, however, I can't run it, because it complains that sun.misc.Cleaner, referenced in Cleanable.java, doesn't exist. I can see that there are 2 versions of Cleanable.java, pre- and post-Java 9, and it looks like the pre-java-9 version ends up being used even though my JDK is newer.
What's the best strategy to create a working Jar file then? I'd be fine with one that doesn't work on Java versions before 9.

Funny how I have two mirror issues one after another: #61 complains that Java 8 doesn’t work :)
Can you try it with plain javac and java? Works for me. If it works for you, then problem is in Gradle

You just need to add Multi-Release: true in MANIFEST.MF.
I think it's a issue with shadow jar, see https://github.com/johnrengelman/shadow/issues/729.


PEP 484 stubs for Click 6.6 available on typeshed

Hi, just to let you know, I added Click support to typeshed which is used by Mypy and pytype for type definitions of third-party libraries that don't themselves provide type annotations.
Here's the relevant stubs: https://github.com/python/typeshed/tree/master/third_party/3.6/click
I created them by hand, following your sources pretty closely, which means they are pretty detailed. If there's anything that's missing or wrong, make sure to bug me on https://github.com/python/typeshed/issues.
Let me know if you have any concerns or questions. Thanks!

LGTM at first glance, I apprechiate the effort!

(I assume there isnt anything to do on our side, so Im going to close this)

I'm a little late to the party, but https://github.com/python/typeshed/blob/master/third_party/2and3/click/core.pyi#L281 should return a typing.List[str], no?


Kong v0.12.2 support

Someone with higher permissions needs to kick the build again. Build must have have failed because of Kong docker image issue which has bee rectified now.

ping @ryadav88


feature request: Open new tabs from page graph

Is it possible to implement opening a file in a new tab with middle click and mod+click on a node in the page graph? Right now new page opens in the same tab
Thanks for the great extension!

I think this is something not possible to implement since the click events in the graph view are not possible to be intercepted, this is because the graph is implemented with canvas elements.


Datepicker returns dirty with an initial value in <script setup>

Reproduction
https://formkit.link/ba56ab8e550cd5204bbb36660d27c908
Describe the bug
Thank you for the amazing work 🚀
I noticed that the Datepicker component returns dirty with an initial value in <script setup>. I'm not sure if is something I should be doing in a more "Formkit way".
Thank you once again 🙏
Environment
• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Ok, this is an interesting one. Strictly speaking it is true that the form is dirty, so I think I need to leave this as "wontfix", but we should provide some additional utilities to make this easier — like a function today() — I’ll create a new issue for that. Here’s what is going on.
The datepicker (by default) outputs a string in ISO3601 format. It automatically sets itself to midnight in of the selected day. So when you provide new Date() as the initial value, it modifies the value to something like 2023-07-21T04:00:00.000Z. This makes the input dirty. Now if you provide an ISO string with a properly truncated date (no microseconds and set to the timezone of the user, or your input is configured to a specific tz and you match that) then it will not be dirty because no mutation to the original value is necessary. For example:
https://formkit.link/5fa52867cab69ddb8d1fca8ff20a427a

@justin-schroeder I took your playground and change it to another date. From this 2023-08-01T00:00:00.000Z to this 1954-10-07T00:00:00.000Z and it gets dirty.
If the day is 1954-10-03 it is not dirty, but then if I go to the day after, 04 it gets dirty 🤔

Hi @justin-schroeder 👋,
Did you get a chance to take a look at this weird scenario? 😅
cc: @andrew-boyd

I looked at the link and changed to 1954-10-07T00:00:00.000Z and it is not dirty, anything I need to do to reproduce outside of that?

@fenilli its flakey, lots of dates it does not return dirty, but some it does. I think it has to do with the users timezone.
@Tragio short answer: no, sadly I have not had time to look into this further, but its still on the list 👍

@justin-schroeder I took your playground and changed it to another date. From this 2023-08-01T00:00:00.000Z to this 1954-10-07T00:00:00.000Z and it gets dirty.
If the day is 1954-10-01 or 02 or 03 it doesn't get dirty, but then if I go to the day after, 04 it gets dirty 🤔
https://formkit.link/11676a7dd2512ad0a2aa096ce9eb0379

This should be resolved at FormKit Pro v0.126.6


Test TransactionDB of RocksDB

What have you changed?
This is an experiment about TransactionDB of RocksDB. It has almost same interface with DBImpl.  But TransactionDB could improve its batch write by enable unordered_write.
What is the type of the changes?

Improvement (a change which is an improvement to an existing feature)

How is the PR tested?

Unit test

Does this PR affect documentation (docs) or should it be mentioned in the release notes?
No
Does this PR affect tidb-ansible?
No

/bench

/bench

/bench


Cannot statically link against Tensorflow library in Golang

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
TensorFlow installed from (source or binary): Binary
TensorFlow version (use command below): 1.4.0

Describe the problem
I'm trying to compile a statically-linked binary of the hello world app against the TF 1.4.0 binary per the golang install & hello-world instructions and am not able to successfully link against the TF library as it reports that ltensorflow cannot be found by /usr/bin/ld in the go build.
The command used is:
go build -a -v -o hellotf --ldflags '-linkmode external -extldflags "-static -L /usr/local/lib"' -x github.com/metral/hellotf
I've tried linking against /usr/local/lib (where libtensorflow lives) using -extldflags, CGO_LDFLAGS, LDFLAGS etc, done sudo ldconfig and the env setup with LD_LIBRARY_PATH and LIBRARY_PATH, but cannot move past this step and always get the error status:
/.gvm/gos/go1.8.3/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: cannot find -ltensorflow

The only related issue I've encountered for this is: https://stackoverflow.com/questions/44428816/tensorflow-for-go-demo-example-run-failed, but that didn't help much either.
That being said, I can dynamically link and build the helloworld go code e.g. go build hello_tf.go and go test github.com/tensorflow/tensorflow/tensorflow/go all work successfully; the issue arises when I try to statically link and cannot link to libtensorflow after it compiles, no matter what settings I try using.
Any help or advice would be greatly appreciated. Thanks!

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

TensorFlow for Go depends on the libtensorflow library for the TensorFlow C API. This is currently built and distributed as a shared library (libtensorflow.so) only, not a static library (libtensorflow.a). So, fully statically linked binaries are not yet possible from the binaries we distribute with releases.
Once https://github.com/bazelbuild/bazel/issues/1920 is resolved, we should be able to build static libraries as part of our release and hopefully that will help.
Till then, you'd have to either live with dynamic linking or figure out how to build a static libtensorflow.a using some other build process (e.g., via the CMake build or something)
Hope that helps.


Specify width and height for Image(remote source)

By analyzing the blame information on this pull request, we identified @lacker to be a potential reviewer.

@facebook-github-bot shipit

Thanks for importing. If you are an FB employee go to Phabricator to review.

Hey, is this change needed? At this point if you are reading the tutorial linearly the reader has not learned about "style" yet, and the text only mentions the "source" prop, so it's a bit odd. We were trying to avoid extraneous styling in the examples.

I tried to merge this pull request into the Facebook internal repo but some checks failed. To unblock yourself please check the following: Does this pull request pass all open source tests on GitHub? If not please fix those. Does the code still apply cleanly on top of GitHub master? If not can please rebase. In all other cases this means some internal test failed, for example a part of a fb app won't work with this pull request. I've added the Import Failed label to this pull request so it is easy for someone at fb to find the pull request and check what failed. If you don't see anyone comment in a few days feel free to comment mentioning one of the core contributors to the project so they get a notification.

@lacker  I know what you mean. But the fact is that "style" is required for Image, otherwise the readers would get nothing to show on their screens. (Somehow the react-native-web-player shows the image because the different implementations between web and RN I guess). So either change to a simpler example which would work with only one prop, or have some words about style here. I am not a native speaker: (, and not so confident with long sentences. So I choose the simplest way to PR, that is, to remind you since issues would probably be ignored.

Arrrr ok. Let's get this in and then I can mess around with the wording later. The bot seems to have rejected it though... lets just try again, if it doesnt work again maybe it needs a rebase.

@facebook-github-bot shipit

Thanks for importing. If you are an FB employee go to Phabricator to review.

Seems already done https://github.com/facebook/react-native/commit/6b2a49e73e52bf40ab82f792e7748aa36847c17f

Oh - I'm confused why the Import Failed tag is on here then. Dunno....

And I forget the space between style and the ending tag...sorry to left it to you.

Optimally, the import failed bot should be removed after import started and pull request closed. Just a tooling thing.


Why doesn't it scrolls the screen when we try scrolling from collapsible view section?

Hi, I fixed this scrolling issue, I created an extended version of this library react-native-collapsible-with-infinite-scroll


Additional shapes for the floorpan library

Request - additional shapes in the Floorplan shape library for:

sliding doors
accordion-style doors
bifold doors
bay windows
swing (double action) doors

Similar to that those at: https://i.pinimg.com/originals/30/ce/35/30ce3516eb95d96570ca1c0b945bc55c.png and https://paintingvalley.com/drawings/sliding-door-plan-drawing-34.jpg

Will be in next release


Move HDB artifacts behavior

Describe the solution you'd like
Calculate changes and document behavior when moving HDB artifacts across packages/projects.
Describe alternatives you've considered
In XSK, moving HDB artifacts may be disabled as this may result in undefined behavior.

This issue is closed as the project is no longer maintained


Issue templates

Is your feature request related to a problem? Please describe.
We need issue templates in our repositories.
Describe the solution you'd like
We can re-use these.
Suggestions are welcome.

The suggested templates contain some specifics for desktop applications. We could make it a bit more generic; otherwise I don't see any further problems.
Your environment:
OS: [e.g. Windows 10 Pro 19045.2965/Linux Ubuntu 22.04.2 LTS/macOS 13 Ventura]
Framework: [e.g. Flutter v3.10.2/.NET 7.0/node.js 18.16.0]
App: [e.g. Syrius v0.0.5/Google Chrome 113.0.5672.126]
Commit hash: [e.g. 58eaa81439a39197dd9080230580fb8b47bcd323]


[cryptolib] API cleanup

Description
This is a tracking issue for a few changes I'd like to make to the cryptolib API before too much stuff starts relying on it and causing compatibility issues. Most of these changes we can live without, but I'm prioritizing this work because it will be much easier to do it now than later and the cryptolib implementation work is about to finish a major milestone, so it would be a good time to fix some tech debt.

[ ] Change ECDSA/RSA signature operations to take message digests instead of raw messages as input (it's possible to build the raw-message interface out of this, but not possible to do the reverse, so this gives the caller more flexibility; note that this is not applicable to Ed25519 since it has a fixed hash function that must be used in a specific way internally)
[ ] Update the AES IV on return (since we only have one-shot AES, this makes it easier for callers to build their own streaming version)
[ ] Make the usage of pointers/const more uniform across API functions
[ ] Introduce word buffers and use them for signatures, digests, MACs, ciphertexts, and DRBG input/output (basically everything except message inputs for hash functions/signatures)
[ ] Separate SHAKE/cSHAKE functions (this was already done in the API doc but hasn't been reflected in the code yet)
[ ] Change the naming of aes_kwp functions to "wrap/unwrap" instead of "encrypt/decrypt" (to match the spec and also avoid someone trying to use it to encrypt normal data)
[ ] Review FIPS requirements for avoiding using keys across different modes and ensure we have enough key types to prevent cross-use (sometimes these are really specific, e.g. you can't use the same key with different padding schemes)
[ ] Change the representation of hardware-backed keys to store diversification data in the keyblob to save some storage space and representation complexity
[ ] Adjust async interface if needed to use OTBN's checksum register (see https://github.com/lowRISC/opentitan/issues/19173)

I've also been considering the following changes, but haven't discussed them much yet, so opinions very much welcome:

[ ] Change key transport interface? (replace blinded_to_unblinded/unblinded_to_blinded/build_unblinded/build_blinded with import_blinded/import_unblinded/export_blinded/export_unblinded, so we don't have unblinded keys that are secretly blinded keys and the usage is a little clearer)
[ ]  Remove specialized RSA/ECC key/signature structs and replace them with simply blinded or unblinded keys that then get unpacked
[ ] Remove synchronous functions for asynchronous operations (these seem a little redundant to me; they just call the async versions internally, and removing them would save code size and API surface area)
[ ] Remove one-shot functions for operations that have an init/update/final interface (same rationale as above for async)
[ ] Change AES-GCM interface to have a more typical streaming mode instead of exposing GHASH/GCTR (I advocated for the current interface previously because streaming GCM can encourage decrypting without checking the tag, but there is enough to GCM besides GHASH/GCTR that I'm now concerned about integration and hardening complexity with the current version)
[ ] Either separate HMAC/KMAC keygen or unify AES/HMAC/KMAC keygen into one "generate symmetric key" function (currently we have AES keygen and a unified HMAC/KMAC keygen, which seems a little inconsistent to me)

CC @moidx @y-srini

Introduce word buffers and use them for signatures, digests, MACs, ciphertexts, and DRBG input/output (basically everything except message inputs for hash functions/signatures)

Just to be sure: these word-buffers will continue to expect their respective underlying data to be provided in big-endian byte ordering, right? E.g., looking at the HMAC functest, a key of 0x00010203... would be encoded into a word-buffer as [0x03020100, ...], assuming a 32-bit little-endian target architecture.
This seems like an important property to keep and codify (if it isn't already). For instance, Tock's in-kernel cryptography APIs will use byte-slices to pass around data and key material; we would want to avoid any spurious allocations and expensive operations for byte-order conversions.

Remove synchronous functions for asynchronous operations (these seem a little redundant to me; they just call the async versions internally, and removing them would save code size and API surface area)

These blocking helpers do make it easier to invoke cryptographic operations on fundamentally synchronous code paths (for instance by avoiding manual allocation of a context struct, etc.). Also, if CryptoLib were to be executed in an isolated context (such as in user-mode), they may reduce the number of context switches required for a given operation.
On the other hand, at least in the context of Tock, most internal cryptography-APIs are fundamentally asynchronous and would primarily make use of the asynchronous functions. Would it be possible to optionally provided these methods (e.g., controlled through a preprocessor macro)?

Just to be sure: these word-buffers will continue to expect their respective underlying data to be provided in big-endian byte ordering, right?

Right, all of these word buffers will maintain the property that, read byte-by-byte, they produce the expected byte stream for the algorithm even if that byte stream is big-endian under the hood.
The motivation for introducing the word buffers is that passing bytes around (including over the bus) is risky from an SCA perspective, since there are only 256 possibilities for the value and attackers can analyze it easily. Of course, it's also a bit faster to use word-aligned writes, but the SCA concern is my primary motivation.

On the other hand, at least in the context of Tock, most internal cryptography-APIs are fundamentally asynchronous and would primarily make use of the asynchronous / split functions. Would it be possible to optionally provided these methods (e.g., controlled through a preprocessor macro)?

The asynchronous operations are only provided for things that are likely to take longer than 5-10ms to complete, e.g. ECC and RSA; quick operations will have the synchronous interface only. My thinking here is that at the point when you're running one of these long operations, the context switch is pretty cheap in comparison. It would also potentially be possible to make them optional, or to add them later on if there's a good justification (easier to remove them now and add them if we need them than to keep them and remove them later when other things might be calling them).

Change ECDSA/RSA signature operations to take message digests instead of raw messages as input
Agree ; (We discussed it before, just adding the notes from the discussion)

The API would have an identifier for the hash function for digest
Either an identifier for raw message/digest in the API; or two separate APIs for raw message & digest - What are your thoughts?
While we are at it, would be interesting to revisit the supported the hash modes.

Change key transport interface? (replace blinded_to_unblinded/unblinded_to_blinded/build_unblinded/build_blinded with import_blinded/import_unblinded/export_blinded/export_unblinded, so we don't have unblinded keys that are secretly blinded keys and the usage is a little clearer)

Agree. When this change is made, a reminder to - add a note on the non-exportable keys (from crypto_key_config_t) for the export blinded keys.

Remove synchronous functions for asynchronous operations
Remove one-shot functions for operations that have an init/update/final interface

Agree on the redundancy part. On the other hand for the crypto lib user, might be convenient/easier to use these one-shot APIs. I would lean on keeping it followed by controlling it through a preprocessor macro as suggested.

Either separate HMAC/KMAC keygen or unify AES/HMAC/KMAC keygen into one "generate symmetric key" function
I guess, unifying them sounds good, with an identifier for key-mode.

Either an identifier for raw message/digest in the API; or two separate APIs for raw message & digest - What are your thoughts?
While we are at it, would be interesting to revisit the supported the hash modes.

I think it would help reduce code size and API surface area to just have one single interface that accepts a digest. If we do need the other version later, it'll be easier to add it than to remove it. And I think in this case we could take hash_mode_t as the digest parameter and support any hash function that is an acceptable length and compatible with the signature scheme.

Remove synchronous functions for asynchronous operations
Remove one-shot functions for operations that have an init/update/final interface

These blocking helpers do make it easier to invoke cryptographic operations on fundamentally synchronous code paths (for instance by avoiding manual allocation of a context struct, etc.). Also, if CryptoLib were to be executed in an isolated context (such as in user-mode), they may reduce the number of context switches required for a given operation.

For the crypto lib user, might be convenient/easier to use these one-shot APIs. I would lean on keeping them ; otherwise to control it through a pre-processor macro, as @lschuermann suggested.

Since both of you agree that these synchronous or one-shot wrappers are useful, let's leave them in for now. It might make sense at some point to offer different compile-time packaging for cryptolib, e.g. "lite" if you want to save on code size at the expense of performance/API convenience. In that case we could filter out these functions for those targets which prioritize code size.


[CLI]: Sync raises KeyError: 'step'

Describe the bug

# one line command
WandbLogger.sync()

# passing optional parameters
WandbLogger.sync(
    id=None,
    n_fine_tunes=None,
    project="gpt-3",
    **kwargs_wandb_init
)

KeyError                                  Traceback (most recent call last)
Input In [283], in <cell line: 4>()
      1 # THIS IS TO SYNC ALL THE PAST TO WANDB
      2
      3 # one line command
----> 4 WandbLogger.sync()
      6 # passing optional parameters
      7 WandbLogger.sync(
      8     id=None,
      9     n_fine_tunes=None,
     10     project="gpt-3",
     11     **kwargs_wandb_init
     12 )

File ~/opt/anaconda3/lib/python3.9/site-packages/openai/wandb_logger.py:73, in WandbLogger.sync(cls, id, n_fine_tunes, project, entity, force, **kwargs_wandb_init)
     69 # log starting from oldest fine_tune
     70 show_individual_warnings = (
     71     False if id is None and n_fine_tunes is None else True
     72 )
---> 73 fine_tune_logged = [
     74     cls._log_fine_tune(
     75         fine_tune,
     76         project,
     77         entity,
     78         force,
     79         show_individual_warnings,
     80         **kwargs_wandb_init,
     81     )
     82     for fine_tune in fine_tunes
     83 ]
     85 if not show_individual_warnings and not any(fine_tune_logged):
     86     print("No new successful fine-tunes were found")

File ~/opt/anaconda3/lib/python3.9/site-packages/openai/wandb_logger.py:74, in <listcomp>(.0)
     69 # log starting from oldest fine_tune
     70 show_individual_warnings = (
     71     False if id is None and n_fine_tunes is None else True
     72 )
     73 fine_tune_logged = [
---> 74     cls._log_fine_tune(
     75         fine_tune,
     76         project,
     77         entity,
     78         force,
     79         show_individual_warnings,
     80         **kwargs_wandb_init,
     81     )
     82     for fine_tune in fine_tunes
     83 ]
     85 if not show_individual_warnings and not any(fine_tune_logged):
     86     print("No new successful fine-tunes were found")

File ~/opt/anaconda3/lib/python3.9/site-packages/openai/wandb_logger.py:162, in WandbLogger._log_fine_tune(cls, fine_tune, project, entity, force, show_individual_warnings, **kwargs_wandb_init)
    160 for _, row in df_results.iterrows():
    161     metrics = {k: v for k, v in row.items() if not np.isnan(v)}
--> 162     step = metrics.pop("step")
    163     if step is not None:
    164         step = int(step)

KeyError: 'step'

Additional Files
No response
Environment
WandB version:  0.13.9
OS: MacOS
Python version: 3.10
Versions of relevant libraries:
openai                        0.26.3
Additional Context
No response

Hi @mertozlutiras, thanks a lot for reporting this! I can see you're integrating wandb with openai. Could you please share with me a code snippet of what are you running before this error raises?

I have a similar issue running Windows 11 with Python 3.11:
openai wandb sync -i  ft-redacted
wandb: Currently logged in as: m<redacted> (<redacted>). Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.13.9
wandb: Run data is saved locally in C:\Users\M<redacted>\wandb\run-20230130_135720-ft-<redacted>
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run ft-<redacted>
wandb:  View project at https://wandb.ai/<redacted>/GPT-3
wandb:  View run at https://wandb.ai/<redacted>/GPT-3/runs/ft-<redacted>
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\<redacted>\AppData\Local\Programs\Python\Python311\Scripts\openai.exe\__main__.py", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "C:\Users\<redacted>\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai\_openai_scripts.py", line 63, in main
    args.func(args)
  File "C:\Users\<redacted>\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai\cli.py", line 527, in sync
    resp = openai.wandb_logger.WandbLogger.sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<redacted>\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai\wandb_logger.py", line 73, in sync
    fine_tune_logged = [
                       ^
  File "C:\Users\<redacted>\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai\wandb_logger.py", line 74, in <listcomp>
    cls._log_fine_tune(
  File "C:\Users\<redacted>\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai\wandb_logger.py", line 162, in _log_fine_tune
    step = metrics.pop("step")
           ^^^^^^^^^^^^^^^^^^^
KeyError: 'step'
wandb: Waiting for W&B process to finish... (failed 1). Press Ctrl-C to abort syncing.
wandb:  View run ft-<redacted> at: https://wandb.ai/<redacted>/GPT-3/runs/ft-<redacted>
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: .\wandb\run-20230130_135720-ft-<redacted>\logs

Hi @mertozlutiras @mkleijn! Could you please share a full code example and so I can try to reproduce this? I'm not getting the same error as you do when running this. Also, did you notice this issue appearing in any concrete version?

I'm finetuning some openai models and then run the WandbLogger.sync() as previously shared. What can I share as a full code example? What do you mean by concrete version?

Hi @mertozlutiras, thanks for explaining this! And sorry for not being clear, with concrete version I mean the version of the wandb package, could you please share with me what version are you using and if you noticed this issue appearing after upgrading to any version or have you been experiencing this for a long time? Thanks!

@mertozlutiras @luisbergua i was able to remedy it just now by running "pip install openai --upgrade" to upgrade from openai-0.26.1 to openai-0.26.4
Maybe some dependency got updated that fixed it, not sure. But my sync to WandB now completed succesfully.

Hi @mkleijn, thanks for sharing this! @mertozlutiras could you please try upgrading openai and see if it works?

@mkleijn since we haven't heard from you we are closing this issue. If you still experience this issue, feel free to re-open this issue.


design note I18n-de (german translation rules & dictionary)

This is the 1st proposal

Closed because this material doesn't belong in the design repo.

Closed because this material doesn't belong in the design repo.


ZEIT → Vercel in README

Removing the Slack badge & updating the company name/links from ZEIT Now to Vercel.

Sorry for the extremely slow response here, just getting around to cleaning all this up.
Replaced by https://github.com/vercel/email-prompt/pull/39.
Thank you as always! 🙏


package for including open source notices

Is there any pub package which can pull out open source licenses of the packages used in flutter app and display them on a screen? Kind of building Open source licenses page in app. Android has oss-licenses-plugin

@neerajyadav
Please try https://pub.dartlang.org
and other channels available at https://flutter.io/community
Github issue is used to track bugs and file feature requests.
Closing this issue now.

Tried pub.dartlang.org already. Didn't find anything like this. I think it will be a good to have package. Almost all the apps have Package License page.


Can't buy/update subscription

Today i wanted to update my home subscription. when i started the process, i've got next error:

However, i'm sure that i have tokens on this account:

in console logs i see next errors:

Working on https://robonomics.app/

Fix https://github.com/airalab/robonomics.app/commit/39ed7ed36dccc0d795c2746f2c42f7fd7b790b2d


Nighly Build - 16.02.2012 - d53b445 - List -> Map

When using list view to show map, it now goes to the default full world size (5300km scale bar in the lower left corner) instead of zooming to a range that would be all inclusive of the caches (list I'm using is encapsulated within about a 15km radius.)  This takes about eight or nine zoom +'s to get down to where it normally opens not to mention the other 3 or so zooms to get to about a 1km scale bar that's more realistically usable.
Not a major issue, but considering the live map isn't fully working yet and I'm using lists w/ about 2K caches to do my caching it would be more convenient.  To not have to zoom so many levels every time I open the map window.

When a map is opened for a cache list the intention is to show a map window that fits all caches. This seems not so reasonable with 2k caches on a list (something I never tried or used myself), so there we may need to revisit the current functionality.
regarding the bug you are reporting: I could not reproduce it.
Could you try to pick two or three of the caches that represent the said radius, move them to a separate list, check that the same behaviour occurs and post the cache codes here? Perhaps I can reproduce it then.

There's already an issue related to the initial zoom level.

As @rsudev already mentioned the initial zoom level if you use the map-function will set the zoom so that you are able to see all caches on your list..Its not a bug but a feature.
@TAZ427 : I suggest to use this "workaround": Divide your pocket queries to a limited geographical area and create several of them (as you like), then import the PQs to different lists (you can create several). So you are easily able to map a certain area with all its caches (e.g. for one city).
@rsudev: But I think that another issue. Map location from a list should always cover all caches.

I checked it with a list that I had w/ 3 caches in it.  It still does the same thing.
Here are the GC codes.  There's about 15-20km separating them and it launches the map at 5300km scale bar.
GC33K80
GC1TTJB
GC2RD1
Regarding the quantity of usable caches in a list, in the past I've been able to open a map of a list of up to 4500 caches w/o issue.  I tried 5440 after that and it crashed, so I stepped back to 2K caches as it gets me about 10mile radius.  I have 8 PQ's running weekly that get all the caches w/i a 65mi radius of Houston, TX.  I dump this into GSAK and can generate GPX files based on my needs.  I wish my Garmin Oregon had this much capacity.  I love using c:geo for getting me to the general area, and reading logs, cache notes, sat views etc.  But for accuracy in the woods, the phone just doesn't cut it.

@Lineflyer If you read my message, I indicated that all caches in the list where within a 15km radius.  I have about 2 dozen lists on my phone w/ caches in Turkey, Germany, UK, China, and various US cities.  The one I was using was my limited local list.

I hit the wrong button before, didn't mean to close.

I will check your issue. Which map are you using (Google, OSM, Offline) ?

Its interesting and seems to be a bug:
I use the two caches mentioned by @TAZ427:
GC33K80
GC1TTJB
and store them in their own list.
Additionally I have some other lists separated to some cities.
If I map the list with these two caches is shows me a zoom level of 5300km/unit.
All my other lists do show up correctly with the zoom limited to the area of caches.
Could it be a bug depending on the cache locations (because they are in the US) ?
Used:
NB1602 - d53b445
Google:Map
BTW: Changed the topic and labeled it.

I've tried it w/ Google, OSM, and Offline
I've also tried it w/ lists I have in Turkey, UK, Germany, China and the Bahama's as well as other US cities.  All lists are no more than 30mile radius and all initialize the map to 5300km legend bar.  lists range in size from from 3 to 4,000 w/ the majority << 1000.
I tried it w/ the various offline maps I've created for my travels - texas.map, turkey.map, shanghai.map, Xian.map, UK.map, Germany.map - don't have a Bahamas.map file


SP 2016 : react web part working in hosted workbench not in web part page

Category :- Bug
I created a new client web part using react in SP 2016 and is working fine in hosted workbench (_layouts/15/workbench.aspx). My web part is making a web-api call using fetch method after user clicks on a button  and displaying results on the page.
Issue is that same web part after adding on a web part page is not working. Page refreshes after clicking on the button, it seems to be failing on the fetch() method. Any idea?

It seems to be an issue with classic SharePoint page. There is already an issue which closed by the group few months back :-
https://github.com/SharePoint/sp-dev-docs/issues/492
It worked for me after pasting following code at end of render method in react web part:-
const buttons = this.domElement.getElementsByTagName('button');
if (buttons && buttons.length) {
for (let i: number = 0; i < buttons.length; i++) {
if (buttons[i]) {
/* tslint:disable /
// Disable the button onclick postback
buttons[i].onclick = function () { return false; };
/ tslint:enable */
}
}
//  }
}


chore: fix vulnerabilities

Rutine run npm audit fix and npm i --save decentraland-renderer@latest

Tackles https://play.decentraland.zone/branch/chore/fixvuln/index.html?ENV=org as per https://github.com/feross/simple-peer/issues/470


Multi-organization support

Currently, when testing the frontend, it is desirable to test mutliple organizations.  The frontend is design to take one and only one organization (even though the claim for organization presents as an array).  This task would create a "fast switch" between organizations on the frontend.

if the current user has DHPrimeAdmin claim for organization - present a dropdown allowing the user to switch organizations.
when making the backend calls, you will need to pass the actual organization used to make the calls (vs. just passing the token and getting the first one)
you will also need to verify that the user's claim allows it make the call (asOrg is in the list of claims for organization).
on the front end - it should remember the user's organization until changed

Hello @mry-usds, cc @mauricereeves
Is this still needed?
Would this be a good first issue for me to tackle?
This seems to involve the whole stack, but is simple enough for a newcomer?
Thank you 🙂

Hi @ahay-agile6 - yes, its a good first issue, currently, Okta is returning all organzations in a user's claim (we just pick the first one) - so it should be relatively easy to display a dropdown.  Toughest part will be the modification to the REST calls to pass the claim (currently it looks up the user's claim on the back end).  If you decide to do this (and need some help) don't hesitate to reach out.

Assigning to myself in order close this out when we close out my OTC-Fixes branch

This looks to be done?


Feature request : configure video quality

I really appreciate the work you put in this and it's a great tool, especially being able to run it in docker.
But i notice the files get big very quickly.
Can you either provide an option in the tool to change the video quality, or provide quidance howto compress the file using ffmpeg for example.

I've updated the new version 0.94 Alpha which adds 3 new arguments
--resolution 240, -r 240           Video resolution, could be 240, `480`, `540`, `720`, `1080` (default: "1080")
--resolution-fallback up, --rf up  Looking for larger or smaller resolution (up for larger, `down` for smaller) if a specified resolution was not found (default: "down")
--fps value, -f value              Preferred framerate, only works if steaming source supports it, otherwise it will always be 30 FPS (default: "60")

Now you can fetch the stream with a specified resolution:
$ ./chaturbate-dvr -u CHANNEL_NAME -r 300

It will looking for 300p video resolution, depends on the value of --resolution-fallback up/down, the program will lookup for nearest resolution. Example output looks like this:
$ ./chaturbate-dvr -u CHANNEL_NAME -r 300

2023/10/01 18:37:51 Found available resolutions: 240, 480, 720, 1080
2023/10/01 18:37:51 Preferred video resolution 300p not found, use 240p instead.
2023/10/01 18:37:51 Fetching video in 30 FPS.

The Dockerfile is a bit too old, you might want to edit it yourself to support the new arguments (or send Pull Request to update it if you can).
Remember to report the bugs if any error occurred.


don't hide the open brace tokens

it makes the parse tree pretty confusing because the parent node will contain a close brace token but no open brace token

Hi, thanks for contributing!
Could you show me an example of the parse tree where you're getting this inconsistency? I don't see why the token wouldn't be available the same way the close token is.

Just to clarify my position: this change will make the parse tree very noisy, and also unbalanced, b/c the opening bracket will be a real node and the closing bracket will still be hidden. I also have not been able to see the issue that you're having (when i use the neovim tree-sitter playground, it includes everything in the tree).
So I would like to see how to produce the issue that you're having, so we can approach the problem together, perhaps from a different angle that won't make the parse tree weird.
Also, if you're commited to this, please update the tests

The nodes can be aliased to '{' whenever used to be consistent with '}'.

@doy thanks for the code! I eventually figured out how to compile + run it (not the most C person in the world), and I think I understand what happened.
With regards to the particular way that you're using it, it looks like the issue is using token in the grammar turns the node into a token node, which then won't get descended into, even though it contains a string node inside.
The simpler solution is to change the definition of _PERLY_BRACE_OPEN to
_PERLY_BRACE_OPEN: $ => alias(token(prec(2, '{')), '{'),

which will make it pretend that it's just a text node, while still being a token.
I still would like to see which queries weren't working, i know i just had to do a similar thing with the PERLY_SEMICOLON node so I could highlight it.

yes, #114 works, thanks! that is a much nicer solution


ReFrame ignores num_cpus_per_task for CompileOnlyRegressionTest

I tried setting num_cpus_per_task = 4 in the class body of a CompileOnlyRegressionTest, but ReFrame ignores it.
I also tried setting it with:
    @run_after('setup')
    def set_cpus_per_task(self):
        self.job.num_cpus_per_task = 4

but apparently there is no job object defined for CompileOnlyRegressionTest, as I get this error:
AttributeError: 'NoneType' object has no attribute 'num_cpus_per_task'

I think ReFrame will take num_cpus_per_task into account only when we set build_locally=False, ie., when we want to build on the remote node. What exactly are you trying here by using num_cpus_per_task for compiling the sources? Maybe you are looking for build_system.concurrency?

@mahendrapaipuri thanks for the reply.
I should have mentioned that I have indeed set build_locally=False to run the build step on the remote node. I've added num_cpus_per_task in order to speed up the build, but that does not work for some reason: only one core is requested/allocated for the job.

self.num_cpus_per_task did not work, but self.build_job.num_cpus_per_task does work (that's probably what you meant?)
as a suggestion, maybe you could automatically set self.build_job.num_cpus_per_task equal to self.num_cpus_per_task if not defined?
anyway, thanks a lot for the help!

I am not sure that is a good idea to always use the RegressionTest.num_cpus_per_task for the build job. It probably makes sense for compile-only test, but not sure if it does for any RegressionTest. On the other hand, the various attributes of the Job class are not documented, while they could, especially now that we have two types of jobs, build jobs and run jobs. I'm keeping this open, until we conclude on a request for enhancement either for the framework or simply for the documentation.

@vkarak The way I understood is RegressionTest.num_* attributes are inherited by ONLY "run" job and not build_job. I think that is why even in the case of  CompileOnlyRegressionTest, setting self.num_* did not have any effect on build_job (from what @smoors told us). Am I seeing it correctly?

@mahendrapaipuri Yes, this is right. I generally believe that we should simply expose the Job attributes so that users can adapt the build_job. After all, this is not the most common operation.

Opened #2315.


[WIP] Improve my Scalafix: RFC

This adds many of the Scalazzi rules. (also sorts imports, but that's a small thing.)
What rules do we want to have. Which ones would we like to keep? Which ones would we like to add?
We would need to translate some rules from scalaz to cats.
Eventually we could migrate away from Wartremover.
Comments and suggestions welcome.
For example, we probably don't want to enforce noDefaultArgs, because default args is essential feature of config classes with PureConfig.
On the other hand, we might want to enforce noValInAbstract, because it could lead to initialization-related runtime bugs.
EDIT: some of the changes are only of whitespace, so I recommend to turn on Hide whitespace changes

disable specific check using comment
yep. with annotation, to be precise. see the diff, I've already did it at some places

It's a lot of rules at once. I would like to understand what we sign up for. So even though I understand it is a lot of work I would like to see a list of rules that are added with some short explanation why we want to add them (unless it's really obvious). For example the thing with covariant and contravariant types makes no sense to me. Disabling java.io also seems very strange - I know it's blocking but blocking does not need to be bad!

Also... what is the relation of import sorting between Scalafmt and Scalafix?
Also, please remove the commented parts - they shouldn't be there and make the PR bigger and more confusing. Thanks.

what is the relation of import sorting between Scalafmt and Scalafix

they complement each other. Scalafmt sorts imports that are on one line (ala import x.{y, z}). Scalafix is sorting the import lines.

@jakubjanecek I open the PR even with commented rules, so that we can have a discussion about them -- otherwise how could we decide if we want them or not.
I personally don't have a clear idea about what rules to enforce and which would be counterproductive.
Some of the rules are described here: https://github.com/scalacenter/scalafix/blob/master/scalafix-rules/src/main/scala/scalafix/internal/rule/DisableSyntaxConfig.scala#L18
Btw, maybe I should have mentioned it in the beginning -- this scalafix config comes from scalazzi and in this PR I would like us to discuss it and transform it into something suiting us.
Regarding variance, we all agree, so I've commented it out. Regarding java.io, can you please put a comment under the rule?

This PR was split into many smaller PRs which were accepted with some changes.


Unclear how to setup ports on Mac OS

I've been diving into Docker today and trying to get an existing app containerized.
I'm running into problems with getting port forwarding setup correctly. My dev machine is a Mac.
The app listens on 8080. In the dockerfile, I have "EXPOSE 8080" enabled.
When I start the container, I use:
"docker run -d -p 8080:8080 appName".

However, I cannot access the server. I've verified the server is running via "docker top".
When I "docker-machine ls" I see:
default   *        virtualbox   Running   tcp://<IP_ADDRESS>:2376

I've been reading various things for a few hours but every reference to ports refers to boot2docker, which according to the Docker site has been deprecated.
Not sure where to go next.
Thanks.

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

I think this section of the documentation describes what you're looking for; Mac OS X - Access container ports
Also, make sure that the service in your container is listening on <IP_ADDRESS>, not <IP_ADDRESS>
However, please be aware that the GitHub issue tracker is for reporting bugs and feature requests, and is not a support forum. It's better to ask this question;

in the #docker IRC channel on freenode
in the docker-users Google Group
in forums.docker.com

I'm going to close this issue, because this is not a bug or feature request, but I hope the hints above will help you further


Getting No Such Method Error while calling an API to display image

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
I am trying to call the api from coin gecko to display images of the crypto coin, current price, trend over the last 24 hours (through spark line chart) and the change in percentage in last 24 hours.
Expected results
The result should look something like this

Actual results
Here is the actual result

Code sample
Code sample
Here is my API Function:

List? coinMarket = [];
  var coinMarketList;

  Future<List<CoinModel>?> getCoinMarket() async {
    const url =
        'https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&order=market_cap_desc&per_page=100&page=1&sparkline=true';

    setState(() {
      isRefreshing = true;
    });

    final response = await http.get(
      Uri.parse(url), headers: {
      "Content-Type" : "application/json",
      "Accept" : "application/json",
    },
    );

    setState(() {
      isRefreshing = false;
    });

    if (response.statusCode == 200) {
      var x =  response.body;
      coinMarketList = coinModelFromJson(x);
      setState(() {
        coinMarket = coinMarketList;
      });
    }
    else {
      print(response.statusCode);
    }
  }

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
<img width="1144" alt="Screenshot 2023-07-17 at 1 09 53 AM" src="https://github.com/flutter/flutter/assets/32162679/0c6240d9-ef54-4cdc-bc27-08a16acb0126">

Flutter Doctor output
Doctor output
[Paste your output here]

@baqirhemani
Looking at the error, it most likely could be causing due to code implementation, so I suggest you to revisit how you are mapping the incoming data from the api. Also, these kind of issues / questions are better suited to be asked on support channels like StackOverflow.
Closing from here, as this doesn't look like a bug in framework but in code implementation. If you disagree, write in comments and I'll reopen it.


React Draggable not working on Mobile

I am not able to drag component on mobile browsers

Same issue on me too

hey I solved the problem and forget to close issue... In my case it was happening because of onClick() event in parent div ..
use onTouchStart event for mobile devices like this
onClick={(e) => handleSomething(e)}
onTouchStart={(e) => handleSomething(e)}

can you please give example


Bug in token repository unit test

The token repository test produces errors in the last line of TokenRepositoryTest.testTokenRepository(). It deletes a token and when it checks if the token is deleted. This check fails because the token is not deleted.
I think this should be solved by flushing session / entity manager. @ArneBinder I think you already implemented this in the security branch?
For now the test is ignored. Before closing this issue we have to remove the @Ignore annotation from the test.

Will be solved by freme-project/FREMECommon/issues/6


Upgrade Reframe + Replace react-typing-animation With typed.js

Replacing the typing lib because:

react-typing-animation has a dependency on babel-runtime because the new Rreframe version uses @babel/runtime
typed.js is vanilla js which gives us more flexibility (to e.g. render to HTML instead of DOM so we can have a faster landing page)

In the meantime I made the typing and the pause a bit slower. It's a minor preference and I'm 100% fine with making the typing as fast as before though.

Ah and upgrade to latest Reframe version to have doNotRenderInBrowser: true work with a ejected browser code. (And some improved misc stuff:))

Perfect :heart: