Perf: Don't create intermediate tuples in normalizer

What was wrong?
THIS IS A WEIRD ONE:
I thought I've found a potential performance issue within our normalizer code but after crunching some numbers, I'm actually not so sure anymore.
Our current code creates intermediate tuples and thereby loop over the data more often than needed. At least, that's how I understand it and we can in fact, prove that getting rid of the intermediate tuples runs faster.
Here's a simplified snipped demoing the performance difference.
With intermediate tuples:
def multiply(x):
    return x * 10

def test():
    msg = tuple(x for x in range(5))

    uncles_hashes = tuple(map(
        multiply,
        tuple(x for x in msg)
    ))
    transaction_roots_and_trie_data = tuple(map(
        multiply,
        tuple(x for x in msg)
    ))

    body_bundles = tuple(zip(msg, transaction_roots_and_trie_data, uncles_hashes))
    return body_bundles

import timeit
print(timeit.timeit('test()', globals=globals()))

Prints: 5.469376810011454
Without intermediate tuples:
def multiply(x):
    return x * 10

def test():
    msg = tuple(x for x in range(5))

    uncles_hashes = map(
        multiply,
        (x for x in msg)
    )
    transaction_roots_and_trie_data = map(
        multiply,
        (x for x in msg)
    )

    body_bundles = tuple(zip(msg, transaction_roots_and_trie_data, uncles_hashes))
    return body_bundles

import timeit
print(timeit.timeit('test()', globals=globals()))

Prints: 4.732703522982774
However! It turns out, that the more elements we have in msg, the number flip and the version with the intermediate tuples does actually run faster. E.g. if we change the code to be msg = tuple(x for x in range(50)), then, the version with the intermediate tuples runs consistently faster. This is surprising to me as I would have thought the opposite should be the case.
Still looking into this...
How was it fixed?
Cute Animal Picture

I wold suggest running those two with profiling enabled to try and see where the performance hit is happening because that is entirely un-intuitive to me.

Below is all conjecture, and might indeed benefit from profiling.

From "performance flipping", I'd guess this has something to do with the fact that
(x for x in msg)

is a generator object (calls to its next() are made from map() and values (pointers?) returned one-by-one), whereas
tuple(x for x in msg)

is an instantiated tuple object (constructed in tuple() and pointer returned (once?) to map()).

There's also the tuple(map(...)) and plain map(...) difference, which is probably a different beast. In other words, there are four cases of tuplification to consider here, not two.

... Also, I don't see a "reversal". This might be a Python version peculiarity. Mine is CPython v3.7.3.
With this glorious script, I get:
# nmsgelems = 5
test00: 3.281730979000713
test01: 3.5057063780004682
test10: 3.575567134001176
test11: 4.169835692000561
# nmsgelems = 50
test00: 14.890971224000168
test01: 15.964820151999447
test10: 15.694290583000111
test11: 17.228873800000656
# nmsgelems = 500
test00:<PHONE_NUMBER>960011
test01:<PHONE_NUMBER>8600128
test10:<PHONE_NUMBER>0300017
test11:<PHONE_NUMBER>0699963

test00() has no tuple() as described above;
test01() has the inner tuple(x for x in msg);
test10() has the outer tuple(map(...)); and
test11() has both occasions.

Using CPython v3.6.8:
# nmsgelems = 5
test00: 5.746762596998451
test01: 6.3236174329995265
test10: 6.438446665999436
test11: 7.349682849000601
# nmsgelems = 50
test00: 24.74597435999931
test01: 25.555854600999737
test10: 25.6410359649999
test11: 27.19204468099997
# nmsgelems = 500
test00:<PHONE_NUMBER>4300044
test01:<PHONE_NUMBER>1099913
test10:<PHONE_NUMBER>1199998
test11:<PHONE_NUMBER>7300006

Still no "reversal".
But note how CPython 3.6 is less performant than CPython 3.7 in this particular task. ;)

@veox Thanks for helping to measure this! It's weird. Meanwhile I have upgraded my OS (unrelated to this haha!) and now I can't reproduce the turning of these numbers either. It's consistently faster!
Also, back then I had opened up a SO post about it and everyone kept saying they don't see the reversal.  Even better. I took your suggestion (with some minor corrections). Maybe you want to review it again because I had to introduce a new type BlockBodyBundle (singular) which is obviously very close to BlockBodyBundles so that might be a discussion point. I can't think of a better name though and I would still prefer it over the unaliased types.


[Question/Feedback]: Create documentation to illustrate the overlap between ALZ-based diagnostics settings and AMBA.

Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Description
Hey Team,
Consider preparing a document that highlights how an Azure Landing Zone (ALZ) deployment, for example the Terraform implementation of Enterprise Scale (ES), manages specific parts of the metrics and logs, and how this integrates with the AMBA project as well as highlight current gaps so one can understand project's current capabilities at a glance.

Thanks @nekdima for your feedback. I will review this internally and will get back you.
@JoeyBarnes it's my understanding that this is about creating a holistic overview of the capabilities of AMBA as a whole and how it integrates with other services. This includes highlighting any gaps and creating a roadmap. I will assign this Issue to you and reach out to you next week to discuss.


Question about heuristics used to detect vulnerable commits from git range

Hello,
For a research study, I'm interested in the algorithm/heuristics used in OSV to determine whether a commit is vulnerable or not.
The OSSF osv-schema outlines the different type of range, the GIT type is one of them. The specification do not mention sufficient details to obtain a similar behaviour that the one implemented in osv.dev.
Impact.py seems to contain the code that populate the "cloud datastore" that contains all the vulnerable commit associated to a vulnerability object.
Impact.py provide heuristics made on git patches https://github.com/google/osv.dev/blob/02af8ac455cec0502a62bd22b86ce9bbe59cb51e/osv/impact.py#L160 , while the documentation only mention an evaluation algorithm that is based on a version comparison operator ( implying for GIT type to reason on parent relation between commits)
Thus, the specification mention only the following information :

It defines a comparison operator for each range type, here for GIT type:

The relation u < v is true when commit u is a (perhaps distant) parent of commit v

It defines the algorithm based on the comparison operator, thus agnostic of the GIT type

func IncludedInRanges(v, ranges)
  for range in ranges
    if BeforeLimits(v, range)
      vulnerable = false
      for evt in sorted(range.events)
        if evt.introduced is present && v >= evt.introduced
           vulnerable = true
        else if evt.fixed is present && v >= evt.fixed
           vulnerable = false
        else if evt.last_affected is present && v > evt.last_affected
           vulnerable = false

      if vulnerable
        return true

  return false

Is there any plan to integrate the evaluation algorithm related to GIT ranges in the specification ? There is no mention of use of  “equivalent commit” i.e git patch id, neither in the relation definition nor in the evaluation algorithm.
For instance we can propose something like that   :
func IncludedInRanges(v, ranges)
  for range in ranges
    if BeforeLimits(v, range)
      vulnerable = false
      events = addEquivalentEvent(range)
      for evt in sorted(events)
        if evt.introduced is present && v >= evt.introduced
           vulnerable = true
        else if evt.fixed is present && v >= evt.fixed
           vulnerable = false
        else if evt.last_affected is present && v > evt.last_affected
           vulnerable = false

      if vulnerable
        return true

  return false

 func addEquivalentEvent(range)
    eventsWithEquivalent = []
    for evt in range.events
        eventsWithEquivalent.add(evt)
    	//If the event is a last_affected event, no equivalent search
    	if evt.last_affected is present
    		continue
		else
			eventsWithEquivalent.addAll(getEquivalentEvents(evt,range.repo))
	return eventsWithEquivalent

It would be really helpful to add the heuristic related to GIT range type in the evaluation algorithm of the OSV spec.
There are some other, part of the implementation that are not clear for me and that prevent me to reproduce on my side your heuristic (I will create other issues) .
Is there any way to access the database where the result of impact.py is saved ? I did not find any way to obtain the database of all the vulnerable commits (and their associated vulnerability) without re-executing the extraction manually of all commits from range, on the source of all repositories.
Thanks,
Sincerely
Romain LEFEUVRE

Hello,
If I'm understanding your questions correctly, I believe  https://github.com/google/osv.dev/issues/1898#issuecomment-1884071754 and https://github.com/ossf/osv-schema/issues/216 attempts to cover them.
Please let me know if you still have questions.

Hello,
Thanks again for your response !
Yes https://github.com/ossf/osv-schema/issues/216 cover my question. I also agree with @zacchiro in the sense that the cherry pick  case might be integrated into the OSV spec.

Is there any way to access the database where the result of impact.py is saved ? I did not find any way to obtain the database of all the vulnerable commits (and their associated vulnerability) without re-executing the extraction manually of all commits from range, on the source of all repositories.

Do you have any plan to create an API to allow read access in your internal database?  I'm looking for a way to access the list of all vulnerable commit of all the affected package of a given vulnerability.
Sincerly,
Romain Lefeuvre

Hi @RomainLefeuvre

Do you have any plan to create an API to allow read access in your internal database? I'm looking for a way to access the list of all vulnerable commit of all the affected package of a given vulnerability.

At this time, we have no plans to create an API to allow such internal database access (this is due to resourcing constraints to produce and maintain such an API).
There are also no plans to publicly export these commits to GCS (this is due to the volume of data that would need to be exported).


[bug] event.block.type conversion error in blockplaceevent

Describe the bug
When you import a block from blockplaceevent, isn't it imported into event.block.type? Why does it always retrun minecraft:air
To Reproduce
Trigger blockplaceevent and output event.block.type
Expected behavior
Originally, the id of the installed block should come out
Screenshots
If applicable, add screenshots to help explain your problem.
Details (please complete the following information):

OS: Windows
Endstone Version: last

Additional context
Add any other context about the problem here.

It is a bit ambiguous here that the property actually represents the block getting replaced. To get the block to be placed, use the "block_placed_state" property.
We design it like this because the event is called before the block is actually placed into the world. So there is no way to get an instance of the block to be placed, instead you can only get a state that will be set to the block if the event is not cancelled.


有linux的桌面窗口客户端吗

有窗口就没那么麻烦，不然每次换节点都要手打编辑文件

QT, but you can use multiple configurations, it's less buggy than QT.

QT, but you can use multiple configurations, it's less buggy than QT.

QT好像不支持ssr，multiple configurations是什么

Use different .json files and write a script to activate or switch b/w them. I didn't realize there's no qt version for ssr but normal configs would work for ss-qt5 (no obfs).


auto import not working properly

Issue Type: Bug
auto import is not working in the new udpate, it was working in the previous build
VS Code version: Code 1.41.0 (9579eda04fdb3a9bba2750f15193e5fafe16b959, 2019-12-11T17:59:52.065Z)
OS version: Windows_NT ia32 10.0.18363
here is a short video on that
https://drive.google.com/file/d/1inQ1yZGLSfxeXmGLcDl81nEsATyTrjik/view

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (8 x 1992)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: enabledrasterization: enabledskia_renderer: disabled_offsurface_control: disabled_offsurface_synchronization: enabled_onvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.90GB (5.95GB free)

Process Argv

Screen Reader
no

VM
67%

Extensions (5)

Extension
Author (truncated)
Version

vscode-apollo
apo
1.12.2

gitlens
eam
10.2.0

prettier-vscode
esb
3.16.1

python
ms-
2019.11.50794

vscode-import-cost
wix
2.12.0

@Tahir500 Please share repo steps and code as text in addition to the videos

@IllusionMH I am talking about direct dependencies. But I just tried using an older TS version with the same result, so maybe this is a vscode issue and not a typescript issue.


[tool_crash] _CastError: (#0      new WebPlugin.fromYaml (package:flutter_tools/src/platform_plugins.dart:547:40))

Command
flutter pub get

Steps to Reproduce

...
...
...

Logs
_CastError: (#0      new WebPlugin.fromYaml (package:flutter_tools/src/platform_plugins.dart:547:40))
#0      new WebPlugin.fromYaml (package:flutter_tools/src/platform_plugins.dart:547:40)
#1      new Plugin._fromMultiPlatformYaml (package:flutter_tools/src/plugins.dart:142:21)
#2      new Plugin.fromYaml (package:flutter_tools/src/plugins.dart:78:21)
#3      _pluginFromPackage (package:flutter_tools/src/flutter_plugins.dart:63:17)
#4      findPlugins (package:flutter_tools/src/flutter_plugins.dart:88:28)
<asynchronous suspension>
#5      refreshPluginsList (package:flutter_tools/src/flutter_plugins.dart:1080:32)
<asynchronous suspension>
#6      FlutterProject.ensureReadyForPlatformSpecificTooling (package:flutter_tools/src/project.dart:357:5)
<asynchronous suspension>
#7      PackagesGetCommand.runCommand (package:flutter_tools/src/commands/packages.dart:179:5)
<asynchronous suspension>
#8      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1257:27)
<asynchronous suspension>
#9      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#10     CommandRunner.runCommand (package:args/command_runner.dart:209:13)
<asynchronous suspension>
#11     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:283:9)
<asynchronous suspension>
#12     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#13     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:229:5)
<asynchronous suspension>
#14     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:64:9)

[✓] Flutter (Channel stable, 3.7.6, on macOS 12.4 21F79 darwin-arm64, locale es-419)
    • Flutter version 3.7.6 on channel stable at /Users/jeanpierrepontesolis/Documents/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 12cb4eb7a0 (12 days ago), 2023-03-01 10:29:26 -0800
    • Engine revision ada363ee93
    • Dart version 2.19.3
    • DevTools version 2.20.1

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/jeanpierrepontesolis/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • ANDROID_HOME = /Users/jeanpierrepontesolis/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13F100
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] VS Code (version 1.68.1)
    • VS Code at /Users/jeanpierrepontesolis/Downloads/Visual Studio Code.app/Contents
    • Flutter extension version 3.60.0

[✓] Connected device (3 available)
    • iPhone 11 Pro Max (mobile) • 6FE73512-D2B6-46A6-B39B-B34F8910DFAF • ios            • com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
    • macOS (desktop)            • macos                                • darwin-arm64   • macOS 12.4 21F79 darwin-arm64
    • Chrome (web)               • chrome                               • web-javascript • Google Chrome 111.0.5563.64

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

Flutter Application Metadata
Type: app
Version: 1.0.1+1
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: stable
Creation framework version: 85684f9300908116a78138ea4c6036c35c9a1236

Closing this as a duplicate of https://github.com/flutter/flutter/issues/115738.
Kindly subscribe to the original issue for updates.
Thank you


Missing ^ Symbol

My password is very secure and it has a ^ in it, which this app doesn't allow. Please fix this.

This app generates passwords itself, what you're typing when you create a password is just the password's nickname which provides entropy to derive the password.


terasology

What you were trying to do
What actually happened
How to reproduce

Step 1
Step 2
Step 3

Log details and game version
Computer details

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/34599285-terasology?utm_campaign=plugin&utm_content=tracker%2F5279&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F5279&utm_medium=issues&utm_source=github).

ayuyu


New selection actions.

Link all spots in the selection with shift-K. This emulates what we had in the popup menu of TrackScheme.

Actions to select all (ctrl A, cmd A), select all spots (ctrl alt A, cmd alt A) and select all links (ctrl shift A, cmd shift A).

Integrated since beta-21


Revert "(PDB-1412) Update rspec dependency from 2.x to 3.x"

Reverts puppetlabs/puppetdb#1344

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/1030/


[3.2b rocksdb] out of memory error when running UPDATE/REPLACE on medium-size collection

my environment running ArangoDB
I'm using the latest ArangoDB of the respective release series:

[x] 3.2 Beta4 using RocksDB

On this operating system:

[X] Linux

[X] Ubuntu .deb

this is an AQL-related issue:
I'm issuing AQL via:

[X] web interface with this browser: Firefox running on Linux Mint
[X] this Driver: arangojs

I've run db._explain("<my aql query>") and it didn't shed more light on this.
The AQL query in question is:
FOR d IN DocCollection
    REPLACE d WITH d IN DocCollection

(UPDATE also gives the same error)
The error is:
out of memory (exception location: /var/lib/otherjenkins/workspace/RELEASE__BuildPackages/arangod/Aql/ModificationBlocks.cpp:246) (while executing) (exception location: /var/lib/otherjenkins/workspace/RELEASE__BuildPackages/arangod/RestHandler/RestCursorHandler.cpp:130)
The collection in question is medium sized, 1-1.5gb with 600mb of indexes (guessing from a similar collection loaded without RocksDB). I'm running it on a computer with 16gb RAM. When I run the query, ArangoDB's RAM usage increases to 5.7gb from 3gb, but there is still ~4gb free when I get the out of memory error. I have 38gb free on my hard drive.
The same query works fine when not using RocksDB.

arangod> 2017-07-18T07:31:01Z [17273] WARNING {queries} slow query: 'FOR d IN test\n    REPLACE d WITH d IN test', took: 2147.509951
2017-07-18T08:03:13Z [17273] WARNING {queries} slow query: 'FOR d IN test\n    REPLACE d WITH d IN test', took: 396.748973
2017-07-18T08:03:13Z [17273] WARNING caught exception in RestCursorHandler: conflict (while executing)
/mnt/c9/jan/.bin/start_rocks_server: line 43: 17273 Killed                  "${cmd[@]}"

When running the reported query my process was killed. I am not sure why. The difference in time between the first and second run looks a bit odd to me. the conflict might be triggered by too fast clicking on execution in the web ui.

Just an update that I still get the same error in the 3.2 final release.

I think "out of memory" is a misleading error message here.
It is probably some other error that is hidden by that message. There are some exception catch blocks which will return "out of memory" regardless of what the actual exception was. This needs to be fixed so we can find the actual error and fix it.

We get the same error in the 3.2 final release. We do replace within transaction in foxx service (RocksDB storage).

My colegue can't edit or delete some record in arangodb own web UI. I can do same in 3.2.beta4, he has 3.2.final, so final version is even worse than beta in our current opinion. We have temporarily switched backed to MMFiles. But we really need to use rocksdb because of many concurrent updates.

What exactly do you mean by "can't edit or delete some record in arangodb own web UI"?
Is there any error shown or logged? Can you elaborate?

Change json value, click save,
PUT XHR http://<IP_ADDRESS>:8529/_db/eex/_api/document/device [HTTP/1.1 202 Accepted 4ms],
no error in web browser console,
no error in ui,
no error in /var/lib/arangodb3/arangod.log,
refresh browser or reopen collection and record,
record has old value ..

@jsteemann I think i finally found it !! We have:

ArangoDB 3.2 final in Docker (official image)
RocksDB engine
fulltext index on some field
I can't modify records that don't have that field. So if we have fulltext index on field fulltext, we can edit records where we pass fulltext field during insert, but if we don't, we can't replace or remove or edit this record.

This is both arangodb web ui (no error, probably ignored) or rest api (out of memory) or foxx (out of memory)

Thanks very much for investigating this.
I'll have a look right now.

Thanks to the very good description I am able to reproduce this.
Working on a fix now.

Just confirming that removing the fulltext index on my test collection stops the error from occurring.

3.2.1 has been released and is available for download.


Gallery: New posting interface and channel loading screen

Revamps the design for block input in Gallery channels (desktop and mobile). Supports drag and drop, paste, and previews for images and refs.
Drag and drop is disabled on mobile, and file upload silently ignores drag/paste if storage isn't enabled.
https://github.com/tloncorp/landscape-apps/assets/90741358/4f7215ae-2838-4f4b-987d-dedce4ad2c25
Fixes LAND-614
Fixes LAND-401

Amazing work. Thank you!
Ran into something minor we might want to either document or refine later. When posting a link + more text, we get a link item with the text appended as expected in the title:
https://github.com/tloncorp/landscape-apps/assets/748181/d3291bfa-1443-44b7-b4e5-6c2f5f9876f2
Yet if you prepend the link with some text, you get a text block with a rich-formatted link (if you follow with a space):
https://github.com/tloncorp/landscape-apps/assets/748181/e3f3a13a-518a-46c8-859d-c4177795cd48
The exhibited behavior is sufficient for the time being and is actually somewhat predictable. I don't know that we need to do anything more or add any more UI to this. It's something to be aware of, though.

I can't repro the drag target showing on resize. Let's merge and see if it crops up on binnec


Log warning when unsupported problem report code is received

resolve #1397

Hm, I don't think this is quite what I had in mind. It's still raising a ValidationError, so the failure would remain. It would probably make sense for the problem report message classes to not validate the code at all (apart from it being a string), and to leave that up to the message handler - just logging a warning when it's not recognized.

Hm, I don't think this is quite what I had in mind. It's still raising a ValidationError, so the failure would remain. It would probably make sense for the problem report message classes to not validate the code at all (apart from it being a string), and to leave that up to the message handler - just logging a warning when it's not recognized.

I guess something like this:
@validates_schema
    def validate_fields(self, data, **kwargs):
        if not data.get("description", {}).get("code", ""):
            raise ValidationError("Value for description.code must be present")
        elif data.get("description", {}).get("code", "") not in [
            prr.value for prr in ProblemReportReason
        ]:
            if len(data.get("description").keys()) >= 2:
                locales = list(data.get("description").keys())
                locales.remove("code")
                LOGGER.warning(
                    "Unexpected error code received.\n"
                    f"Code: {data.get('description').get('code')}, "
                    f"Description: {data.get('description').get(locales[0])}"
                )

Yes, I think that would work


Random  never  return 0

In util/ random.h, we are computing  seed_ = (seed_ * A) % M,    where M =
2^31-1, and the seed_ cannot be zero.

So, Random::Next() function never returns zero.

The member function of this class

 uint32_t Uniform(int n) { return Next() % n; }

returns a uniformly distributed value in the range [1..n-1] , not [0..n-1]

patch:
change the last line of  function Next()  from
 return seed_;
to
 return seed_ - 1;

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 29 Aug 2013 at 4:25

Random::Next() may never return 0, but it can return n and 2*n and 3*n etc.  As
a result the values returned are [0..n-1] as stated.  For values of n < sqrt(m)
the "uniformly distributed" clause is also valid.

Original comment by<EMAIL_ADDRESS>on 23 Oct 2013 at 5:02


nixos-generate-config: enable tp_smapi for old ThinkPads

See: http://www.thinkwiki.org/wiki/Tp_smapi

"If you are installing on a recent Thinkpad that has an Ivy Bridge or newer processor (X230, T430, T530, etc.), tp_smapi will not work."
Although X220 seems to be quite popular here

Since it's hard to detect all the possible hardware correctly I'd rather delegate this to the user. Maybe it could be worth mentioning  nixos-hardware and its usage in the NixOS manual

It's rather easy, and the proper place to do it is in nixos-generate-config. We already do that in nixos-hardware, and it's less than ideal.

I meant in general, not in the specific case, for example in #3303 I can't think of a reliable way to detect if the system is a laptop or not.
I agree that the place to generate a configuration according to the underlying system is nixos-generate-config, now that I read the source I recognize that it already does hardware recognition. Maybe it's worth looking into a way to integrate it with the knowledge inside nixos-hardware.
Without going too off-topic, I have a X200 with libreboot on it, so tp_smapi does not work. Is there a way to detect this condition?

I meant in general, not in the specific case, for example in #33033 I can't think of a reliable way to detect if the system is a laptop or not.

cat /sys/devices/virtual/dmi/id/chassis_type

List of chassis type IDs:
https://github.com/jhcloos/flashrom/blob/5ccc913c9d445b4c9e5611b2fa54bb73ea7c9676/dmi.c#L79

Without going too off-topic, I have a X200 with libreboot on it, so tp_smapi does not work. Is there a way to detect this condition?

I envy you. Could you show me your dmesg?

cat /sys/devices/virtual/dmi/id/chassis_type

The more you learn...

Could you show me your dmesg?

Sure, sent it via email

And, as far as I can tell, the table that shows the meaning of the integer values in /sys/devices/virtual/dmi/id/chassis_type is here:
http://www.dmtf.org/sites/default/files/standards/documents/DSP0134_3.1.1.pdf
(Look for "System Enclosure or Chassis Types".)
An alternative is using dmidecode, if you have permissions:
$ sudo dmidecode -s chassis-type
Desktop

@zarelit: At the very least it's possible to identify that you are using coreboot, which is roughly the same as libreboot plus some blobs: https://github.com/coreboot/coreboot/blob/1a9035988addfbe89ed708fe3c91bf6cc54befdb/src/drivers/i2c/at24rf08c/lenovo_serials.c#L199
Match against DMI in dmesg and it will be there. coreboot blobs do not include anything ThinkPad-related, so behavior (and bugs) should be identical to libreboot in this case, including defunct tp_smapi.


Arduino Motion Sensor

Is your proposal related to a problem? Please describe.
Add an Arduino setup that detects motion and lights up a LED as an indicator.
Add any other context or screenshots about the proposal request here.
Issue made under GSSoC'22 contributions.

@Kushal997-das Kindly assign me this issue to work on. same with #517


Mapping recommendations

Consolidated environment considerations from several pages into the right spot. Crosslinked from the duplicate areas into the right article.
Adding documentation on how to best map a space with a HoloLens.
This guidance should help people run into fewer hologram quality issues down the line.

Docs Build status updates of commit 3706ca2:
:warning: Validation status: warnings

File
Status
Preview URL
Details

mixed-reality-docs/voice-design.md
:warning:Warning
View
Details

mixed-reality-docs/TOC.md
:white_check_mark:Succeeded

mixed-reality-docs/environment-considerations-for-hololens.md
:white_check_mark:Succeeded
View

mixed-reality-docs/spatial-mapping-design.md
:white_check_mark:Succeeded
View

mixed-reality-docs/use-hololens-in-new-spaces.md
:white_check_mark:Succeeded
View

mixed-reality-docs/voice-design.md

Line 4: [Warning] Invalid value for author: 'shentan' is not a valid GitHub ID

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

I have no idea why the scanner picked up on voice-design.md, but it is not relevant to these changes.

Docs Build status updates of commit 7ef549e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/TOC.md
:white_check_mark:Succeeded

mixed-reality-docs/environment-considerations-for-hololens.md
:white_check_mark:Succeeded
View

mixed-reality-docs/spatial-mapping-design.md
:white_check_mark:Succeeded
View

mixed-reality-docs/use-hololens-in-new-spaces.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Is documentation site down?

🐛 Bug Report
https://kulshekhar.github.io/ts-jest is not working saying 404 in github. Something got changed to deploy the pages?
To Reproduce
Steps to reproduce the behavior:
Expected behavior
https://kulshekhar.github.io/ts-jest should show ts-jest documentation
Link to repo (highly encouraged)
https://github.com/kulshekhar/ts-jest/
envinfo
System:
    OS: MacOS

Strange, yesterday I saw it was still working with latest changes

Documentation site shows 404 for me:

Did you guys changed the documentation generation process? I can see that the branch gh-pages is not there now.

Indeed i don't see that branch either. That branch has to be created by owner right ?
cc @kulshekhar

Yes. In settings for a repository for sources, you can choose master/docs folder too. But I am not sure if some custom logic for Jekyll was used or not. If it works as a normal Jekyll template it should work without any problems.

This is what the settings part of the repo shows:

Looks like it might need to be built properly and then pushed to Github (instead of relying on Github to do the build)

I made a few attempts in local today to build this but still not manage to build it.

Same thing here, still down.

I've manually generated the site and switched to using the gh-pages branch for now

I made a few attempts in local today to build this but still not manage to build it.

@ahnpnl Were you getting some errors building locally?
Because I tried locally and was able to build successfully and was able to see updated changes in _sites folder in docs.

I got errors related to gem dependencies. Some can’t be installed for some reasons 😕

I researched a little and found that plantuml is not supported in github at present. Here is the relevant issue for this: https://github.community/t5/How-to-use-Git-and-GitHub/Support-UML-diagrams-in-markdown-with-plantuml-syntax/td-p/3844
It works in my local because I have plantuml utility installed but fails on github due to lack of plantuml support.
One way to fix this is generate documentation locally and then commit the generated output in docs folder and disable jekyll builds by adding .nojekyll file in docs. But this breaks layout as CSS is that good as it was previously,  I tested on my test repo.
Another way is the one being used at present i.e. pushing to gh-pages branch.
I think its fine for now till github comes with plantuml support or the dependency of plantuml is removed from the repository.

Actually if we change the structure of docs folder a little.
We keep md files in docs_md folder and generate jekyll output in docs folder and then tell github to not run jekyll by adding .nojekyll in docs folder. Since we are generating it locally github will use the generated files and will show proper site.
And example of the same: https://anshulguleria.github.io/ts-jest/
You can view these changes in my test commit.
Main changes are only in _config.yml file. Rest all are generated changes.
Let me know if this approach is fine, I can raise a PR for the same with some changes in build step to generate this on publish.
@ahnpnl and @kulshekhar

this sounds good to me. However, is it easy for contributors to generating locally ? Like I have issues with local that I can't run build at all for docs. Can CI help us in this case ? I mean the main goal is making it easy for contributors.

I agree. The only problem I see is plantuml plugin. I will try if the installation can be automated or not. If I succeed a github action can be written for the same.

Found  a way to install plantuml on github actions. Raised a pull request for the change:
PR: https://github.com/kulshekhar/ts-jest/pull/1522
I am still going by using gh-pages branch to avoid unnecessary commits in master branch.
The above mentioned change creates new bot commit on gh-pages branch whenever something changes in master branch.
@ahnpnl Let me know if this is fine.

I like the idea 👍


Case sensitive

File suffix is case-insensitive and cannot match MD5 checksum

Can you be more specific? Not sure what you are trying to say.


Question about constrained problem solving.

Hello, everyone!
I am troubled by a problem and want to ask for your help. I'm doing some research with neural networks, and the objects I'm studying have some constraints (both equality and inequality constraints). Optimizers like Adam only work for unconstrained problems, what can I do to elegantly handle these constraints (I tried the penalty function method, but it doesn't seem to work very well)?

Unfortunately, we don't have support for constrained stochastic solvers at the moment. Our constrained solvers are for the full batch setting. If your constraints are not too complicated, maybe there is a change of variable to rewrite the problem in unconstrained form? For instance, if the constraints are {w : w >= 0, sum(w) = 1}, you can use the change of variable w = softmax(v) and minimize w.r.t. v instead.

@mblondel Thank you for your reply! It helps me a lot!


move rp checks to script based to avoid network

This PR drops external IP address and DNS from RP nodes and moves to script based health checking using RunCommand

@jim-minter tried to use PID and conmon capability to drop in PIDS[1], but this does not give an indication if RP is healthy.
So moved to script-based approach and using the first VM instance as jump-box to check all instances. This way it is much faster than 3 commands, and one call blocks execution until it responds. WDYT?
1 https://www.redhat.com/sysadmin/podman-shareable-systemd-services

reviewed over Slack.


[FR] FR-009: Inject process information into TCP Options using a kernel module.

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
Inject process information into TCP Options using a kernel module.
Obtain process information corresponding to TCP packets without using eBPF.
Scan the /proc directory to establish a socket-to-process mapping, and query the mapping to obtain process information. This works well for long connections, but may not be able to obtain process information for short connection scanning.
Carrying process information (process number) in the TCP packet can solve the problem of short connection.
Inspired by TOA, the process number can be added to TCP Options through the kernel module.
Use case
No response
Related issues
https://github.com/deepflowys/deepflow/issues/1481
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

We implemented the first version of TCP Option Tracing (a.k.a TOT): https://github.com/deepflowys/tcp-option-tracing
Related documents need to be supplemented.

After installing the TOT kernel module, both the client and the server will inject their own IP, PID, and TCP SEQ (use 16 byte) into the TCP Option. DeepFlow can use IP + PID to generate a complete application topology, which is suitable for both short and long connections. In addition, through TCP SEQ, we can also solve the problem of incomplete distributed tracing caused by the possible modification of the original SEQ by LVS.

We have implemented the kernel module and ebpf program, but still need more time to integrate into the topology calculation logic of deepflow-agent.


Update codecov yaml to ignore test files

Summary of changes (Bug fix, feature, docs update, ...)

Fix our coverage levels on codecov.
I'm not really sure how we can test this, if I run the coverage tool locally I get 76% coverage so we'll see how we go.

Please check if the PR fulfills these requirements

[ ] Closes #621

Closing this as we need to look at codecov again.


fix(webhook): typing webhook request handler

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

2 of 4 (50.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 91.078%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/webhook/index.ts
2
4
50.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
974

Relevant Lines:
1046

💛  - Coveralls


Sample count plot updates

Only show exact matches and excluded count, not fraction in ARG
Show exclusion fraction on it's own axis, as it should always be near zero.
Show scorpio composition of samples processed and accepted (in ARG + exact matches)

See also #403

Closed in #417


Rasa docs: Syntax error in "Tutorial: Building an Assistant"

Description of Problem:

Under the Business logic subsection, there is an error in the return type definition for the slot_mappings method.
def slot_mappings(self) -> Dict[Text: Union[Dict, List[Dict]]]:

The colon syntax is used for slices and not for type definitions. The method as written throws an error at runtime.
Overview of the Solution:

It should be corrected to the following:
def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict]]]:

Examples (if relevant):

NR
Blockers (if relevant):

NR
Definition of Done:

Syntax error fixed.

Thanks for raising this issue, @melindaloubser1 will get back to you about it soon✨
Please also check out the docs and the forum in case your issue was raised there too 🤗

Thanks for pointing that out! It should be fixed by https://github.com/RasaHQ/rasa/pull/5345


multiple terminals

allow multiple launched debug sessions - #46

I need to try to use what is difference.

This maps debug sessions to terminals. It still creates terminals named "rdbg" but will create more than 1 if necessary. It will still reuse and open "rdbg" terminals too.
You can try it out by just launching the same debug task more than once. (vscode will warn you, but just proceed)

rebased on master

I also included some eslint fixes and changed terminal name to Ruby Debug Terminal (e9ed034) to match the JavaScript Debug Terminal that ships with vscode. I can remove the name change if you wish.

Thank you!


generalize contact partitioning to flexible age, risk structure

https://github.com/eho-tacc/episimlab/blob/2e34d96ba2f61ba69580cdb10fe279af361eb47e/src/episimlab/partition/toy.py#L65
Toy contact partitioning had two example age groups and no explicit risk groups. Refactor to allow for this flexibility in the contact matrix.
@eho-tacc I can't assign issues in your repo, but I will handle this one :)

resolved in https://github.com/eho-tacc/episimlab/pull/23


Divide by zero with --tiebreak=end for long-line inputs

Checklist

[X] I have read through the manual page (man fzf)
[X] I have searched through the existing issues
[X] For bug reports, I have checked if the bug is reproducible in the latest version of fzf

Output of fzf --version
0.53.0 (c4a9ccd)
OS

[X] Linux
[ ] macOS
[ ] Windows
[ ] Etc.

Shell

[X] bash
[ ] zsh
[ ] fish

Problem / Steps to reproduce
Running
seq -s ' ' 100000 | fzf   --tiebreak=end --query 1

will produce this error:
panic: runtime error: integer divide by zero

goroutine 27 [running]:
github.com/junegunn/fzf/src.buildResult(0xc00027b000, {0xc00001a198?, 0x5?, 0xc0000554e8?}, 0x24)
github.com/junegunn/fzf/src/result.go:84 +0x47e
github.com/junegunn/fzf/src.(*Pattern).MatchItem(0xc0000c2000, 0xc000055608?, 0x10?, 0x10000000001?)
github.com/junegunn/fzf/src/pattern.go:312 +0xa5
github.com/junegunn/fzf/src.(*Pattern).matchChunk(0xc0000c2000, 0xc00027b000, {0x0?, 0x7dd249067108?, 0x7dd24905e5b8?}, 0x18?)
github.com/junegunn/fzf/src/pattern.go:294 +0xf6
github.com/junegunn/fzf/src.(*Pattern).Match(0xc0000c2000, 0xc0000b8450?, 0xc0000834c8?)
github.com/junegunn/fzf/src/pattern.go:281 +0xa6
github.com/junegunn/fzf/src.(*Matcher).scan.func1(0x0, 0x0?, {0xc0000b6028, 0x1, 0x0?})
github.com/junegunn/fzf/src/matcher.go:184 +0x178
created by github.com/junegunn/fzf/src.(*Matcher).scan
github.com/junegunn/fzf/src/matcher.go:179 +0x578

Thanks for the report. Fixed on master.


JSONDecodeError

I was trying to run the bot and check it out. first there was an error in config file JSON formatting ( missing comma after "adminUserId": "--userid--" ). after that the following error was displayed when running the bot :
Traceback (most recent call last):
File "bot.py", line 6, in
import commands as cm
File "/root/cryptotrader-telegram/commands.py", line 11, in
import charts
File "/root/cryptotrader-telegram/charts.py", line 17, in
from lib import app
File "/root/cryptotrader-telegram/lib.py", line 16, in
from config import configData
File "/root/cryptotrader-telegram/config.py", line 11, in
configData = DotMap(json.load(open('config/config.json','r')))
File "/usr/lib/python3.6/json/init.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 14 column 5 (char 418)
any suggestion on this would be appreciated.

Hi,
There seemed to be some comma issues in the config file, I have corrected those and also arranged the config a bit.
Please try again.
Thanks,
Nikhil

Hi,

Hi,
thanks for the fast update. the problem is solved and bot started.
it's currently not accepting commands and displays  "  ERROR error returned: module 'time' has no attribute 'time_ns' ".

Hi,
I have been running this in Docker successfully, could be an error due to Python version.
You tried in Docker?
Regards,
Nikhil

I just tried running this with docker. this time the error changed to:
2020-06-14 12:43:13 MainThread INFO submitting task: schdinfo []
2020-06-14 12:43:13 Command_0_1 ERROR HTTP Error 400: Bad Request
Traceback (most recent call last):
File "/app/commands.py", line 40, in wrapped
app.send_msg(msg, task.message.chat.id)
File "/app/lib.py", line 153, in send_msg
with urllib.request.urlopen(url) as response:
File "/opt/conda/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/opt/conda/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/opt/conda/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/opt/conda/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/opt/conda/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/opt/conda/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
the bot is still running after this error. all the telegram bot commands display access denied message.
thanks for the reply.

Hi,
This is happening because you didn't set the Chat ID for the telegram, which is different from the UserID.
Could you please try sending some commands from Telegram and see if it works ?
I will add some code to notify the user about the error.
Regards,
Nikhil

Please pull the new code and have a look at the readme to see how to find out adminChatId.

Hi,
I pulled the new code and configured the chat id. The " ERROR HTTP Error 400: Bad Request " is gone but still displaying " access denied " error in telegram bot when sending commands. the python bot receives the command like " 2020-06-15 05:17:37 Telegram INFO submitting task: now ['now'] " but nothing else happens and the telegram bot just throws access denied.

Hi,
The "access denied" appears if your adminUserId is not configured with what you have been sending messages with. Please confirm that.
e.g.  My configuration looks like:
"adminUserId": "nrathi*****"

Hi,
Because "Id" was mentioned in "adminUserId" , I ended up entering my telegram id instead of username. after changing this, the access denied is fixed and the bot is working.
thanks you for helping me setting up the bot.


Refactor surveys and add a new one

For #4726

[ ] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
[x] Title summarizes what is changing.
[x] Has a news entry file (remember to thank yourself!).
[ ] Appropriate comments and documentation strings in the code.
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[ ] Unit tests & system/integration tests are added/updated.
[ ] Test plan is updated as appropriate.
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed).

Codecov Report

Merging #4794 (70c8e0f) into main (7276e65) will decrease coverage by 2%.
The diff coverage is 73%.

@@          Coverage Diff           @@
##            main   #4794    +/-   ##
======================================
- Coverage     76%     74%    -3%
======================================
  Files        406     410     +4
  Lines      26741   26943   +202
  Branches    3860    3905    +45
======================================
- Hits       20534   20083   -451
- Misses      4610    5302   +692
+ Partials    1597    1558    -39

Impacted Files
Coverage Δ

src/client/api/types.ts
100% <ø> (ø)

src/client/common/types.ts
100% <ø> (ø)

src/client/datascience/common.ts
73% <ø> (-1%)
:arrow_down:

...active-common/intellisense/intellisenseProvider.ts
74% <ø> (+<1%)
:arrow_up:

...ent/datascience/jupyter/jupyterNotebookProvider.ts
75% <ø> (ø)

...science/jupyter/liveshare/guestJupyterExecution.ts
73% <ø> (ø)

...datascience/jupyter/liveshare/hostJupyterServer.ts
83% <ø> (-2%)
:arrow_down:

...lient/datascience/jupyter/liveshare/serverCache.ts
83% <ø> (ø)

...atascience/notebookStorage/nativeEditorProvider.ts
81% <ø> (+1%)
:arrow_up:

...t/datascience/telemetry/interpreterCountTracker.ts
85% <ø> (ø)

... and 145 more


[BUG]  Invalid descriptor on #imblocker:imblocker.mixins.json:ftbquests.MixinQuestsTextBox

imblockerfabric-1.0.21
ftb-quests-fabric-1902.4.16-build.235
1.19.2 quilt
Describe the bug
org.spongepowered.asm.mixin.injection.throwables.InvalidInjectionException: Invalid descriptor on #imblocker:imblocker.mixins.json:ftbquests.MixinQuestsTextBox from mod imblocker->@Inject::presetFocused(ZLorg/spongepowered/asm/mixin/injection/callback/CallbackInfoReturnable;)V! Expected (ZLorg/spongepowered/asm/mixin/injection/callback/CallbackInfo;)V but found (ZLorg/spongepowered/asm/mixin/injection/callback/CallbackInfoReturnable;)V [INJECT Applicator Phase -> #imblocker:imblocker.mixins.json:ftbquests.MixinQuestsTextBox from mod imblocker -> Apply Injections ->  -> Inject -> #imblocker:imblocker.mixins.json:ftbquests.MixinQuestsTextBox from mod imblocker->@Inject::presetFocused(ZLorg/spongepowered/asm/mixin/injection/callback/CallbackInfoReturnable;)V]

ImblockerFabric.zip
Try this version plz.


Vercel Deployment Not Loading Swiper CDN

Description:
When deploying to Vercel, the Swiper CDN is not being executed, causing the Swiper component not to function properly on the deployed site. This issue is not present in the local development environment.
Steps to Reproduce (if applicable):

Deploy the project to Vercel.
Navigate to the page with the Swiper component.
Observe that the Swiper component is not functioning as expected.

Expected Behavior:
The Swiper CDN should be executed, and the Swiper component should function properly in the deployed environment.
Actual Behavior:
The Swiper CDN is not being executed, resulting in the Swiper component not functioning in the deployed environment on Vercel.
Screenshots or Logs (if applicable):

Environment (if applicable):

OS: N/A (Issue is with Vercel deployment)
Browser: N/A (Occurs across all browsers)
Version: N/A

Additional Context:
The current Content-Security-Policy in the Vercel configuration may be blocking the Swiper CDN from being executed. Here is the relevant part of the configuration:
{
  "key": "Content-Security-Policy",
  "value": "default-src 'self'; base-uri 'self'; form-action 'self'; frame-src 'self'; frame-ancestors 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src 'self' data: https://images.unsplash.com; connect-src 'self'; object-src 'none'; upgrade-insecure-requests; block-all-mixed-content"
}

Issue Explanation
The issue was resolved by adjusting the HTTP headers configuration in the vercel.json file. These headers are crucial for handling CORS (Cross-Origin Resource Sharing) and enhancing the security and performance of your website. Below is an explanation of the importance of each configured header:
Content-Security-Policy (CSP):
This header defines restrictions on the resources that can be loaded on your page. In this case, it allows resources to be loaded only from the same domain ('self') and from https://images.unsplash.com. It also allows inline scripts ('unsafe-inline') and script evaluation ('unsafe-eval'). It's important to note that the initial CSP was restricting the execution of scripts from our CDN, preventing Swiper.js from working correctly. By adjusting this policy, we allowed the necessary scripts to be loaded from trusted external sources.
Permissions-Policy:
Controls the browser APIs and features available to your site. In this case, it is empty (interest-cohort=()), which should not affect the execution of your CDN.
Referrer-Policy:
Defines how the Referer header is sent when navigating between pages. Here it is set to no-referrer-when-downgrade, meaning the Referer will only be sent when the navigation is secure (e.g., from HTTPS to HTTPS).
X-Content-Type-Options:
Helps prevent MIME type attacks. It is set to nosniff, instructing the browser not to guess the content type based on the file extension.
X-Frame-Options:
Protects against clickjacking attacks by controlling whether your site can be displayed within a frame (iframe). Here it is set to SAMEORIGIN, allowing it to be displayed only on the same domain.
X-XSS-Protection:
Helps prevent cross-site scripting (XSS) attacks. It is set to 1; mode=block, enabling XSS protection in the browser.
Cache-Control:
Defines how resources should be cached. Here it is set to public, max-age=0, must-revalidate, indicating that resources can be publicly cached but must be revalidated before use.
Strict-Transport-Security (HSTS):
Enhances security by enforcing secure (HTTPS) connections. Here it is set to max-age=31536000; includeSubDomains; preload, indicating that HSTS should be applied for one year, including subdomains, and be preloaded in browsers.
Solution
To resolve the issue of Swiper.js not loading correctly in the Vercel deployment, the Content-Security-Policy policies in the vercel.json file were adjusted to allow script loading from the Swiper CDN. This ensured that the necessary resources could be loaded without undue restrictions, allowing the slider functionality to work correctly.

In summary, these headers are essential for the security and performance of your website. Adjusting them appropriately allowed Swiper.js to execute correctly in the production environment.


Template Result injected into wrong part of dom

Bug
When combined with custom elements, lit-html injects the template result into the wrong element on the page. Tested on Chrome 69.0.3497.100 and Firefox 63.0b13 Windows 10 64-bit.
Steps to replicate:
repo
npm i
npm start

Open file dist/index.html in a browser supporting custom elements.

Where is the template rendered?

I'm looking into this issue at the moment

I found the problem. This example modified the LightDOM in the constructor of a custom element. This is forbidden by the Custom Element specification, so this is not necessarily an issue. However, simple elements that modify their LightDOM in the constructor can technically exist, and when you use such an element inside a template, it throws off the ability of lit-html to parse the template.
Here is a minimal example:
https://codepen.io/ruphin/pen/oawzNG?editors=1010
The example works fine when we modify the invalid element to render in the connectedCallback instead.
Do we want to modify lit-html to detect this problem somehow, or simply ignore it and point to the Custom Elements spec?

Didn't realize it's forbidden. In that case, I don't think it should be addressed in lit-html.

It would probably save a lot on future support issues to not that limitation in the documentation.

@abraham not sure where. lit-html docs don't really reference Web Components concepts, though we can add links to webcomponents.org or MDN.

@justinfagnani I would put it right on lib/render. I would think a substantial amount of lit usage will be in web components. Or wait until the next support request.

If it's forbidden too, then ignore this, but I noticed that this also happens when modifying lightDOM via attributeChangedCallback. I've updated the repo and here is the codepen version.

AttributeChangedCallback is called before the element is connected, so it's still somewhat weird, but it isn't explicitly disallowed. It is certainly a gray area, but it seems to me like lit-html should be able to handle this case. It might be some regression in the template handling, causing elements to upgrade too soon. I will look into it. Thanks for the report @Toxyxer

This might be a serious issue.
According to the Web Component specification the limitation of not accessing the children or attributes only applies to the constructor. Technically, making changes to the children of a Custom Element in the attributeChangedCallback is allowed, although the spec is not explicitly clear on this. Regardless of whether it is allowed or not, there is a chance users will do this by accident, and it will break lit-html.
Currently we default to using importNode when cloning templates: https://github.com/Polymer/lit-html/blob/d2be7ca48780120be3a66101e0db6a034592c002/src/lib/template-instance.ts#L62-L64
The reasoning behind the clause was that CustomElements should technically not modify their innerHTML before the connectedCallback is called. We had to make an exception for when the Custom Elements polyfill was used, because that is more likely to modify LightDOM when combined with the ShadyDOM polyfills.
Here's a simple reproduction that shows attributeChangedCallback is in fact called when importNode is used: https://codepen.io/ruphin/pen/mzBzOa?editors=1010
Considering this, I think we may have to also use cloneNode with native Custom Elements. Downside is that template cloning may be slightly less efficient when we clone and adopt the fragment in separatele calls. Upsides are that we fix this issue, and lose a special case for polyfills. What do you think @justinfagnani ?
I can make a PR that implements this change and adds a test case with a Custom Element that contains this behaviour, to avoid regression.

This test shows that the attributeChangedCallback has the same behaviour as the constructor with regards to the child content not being accessible.
https://codepen.io/ruphin/pen/dgVgJq?editors=1010
It could be that the intended behaviour of the spec is that the attributeChangedCallback should have the same restriction as the constructor with regards to accessing child content.
I will raise an issue in with the spec to clarify. If the outcome is that attributeChangedCallback is indeed not allowed access, then we are spec compliant, which may change decision process on this issue.

https://github.com/w3c/webcomponents/issues/767

Well, we got confirmation from the webcomponents spec authors that it is fine spec-wise to manipulate LightDOM in the attributeChangedCallback. This means that our current approach of upgrading elements before parsing the tree to find parts is not viable, because upgrading elements may cause attributeChangedCallback to fire, which in turn may modify the DOM, which breaks our code.
I will make a PR that fixes this. Thanks again @Toxyxer for bringing this issue to our attention.

@ruphin I'm not sure we want to incur the extra tree-walks involved in adoptNode and upgrade without making sure it doesn't impact perf too much.

There should be no additional tree walking compared to previous implementation. The only difference is that we defer upgrading elements until after we walk the tree. Can you clarify how this would lead to extra tree-walks?

I will prepare a jsperf test on this to check if there is a significant difference.

Right now we have either 3 or 5 tree walks:
Native Custom Elements:

Import template (native)
Prepare template (JS)
Append fragment (native)

Polyfilled Custom Elements:

Clone template (native)
Prepare template (JS)
Adopte fragment (native)
Upgrade fragment (native)
Append fragment (native)

Changing to use the same approach for each would add 2 native tree walks for the native case. It could be that native tree walks are fast enough that they don't show up, or only show up in micro-benchmarks and not in application code. That would be ideal, as it's the more correct ordering that can handle all sorts of ill-behaved components.

It's true that we need a separate cloning and adoption phase compared to the original single import, but I assume that the adoption tree walk also handles the upgrades without requiring an additional tree walk, similar to the import tree walk.
I think these native tree walks should be efficient enough, but to make sure I will prepare some kind of benchmark to test this.

afaik, this is the correct steps in the correct order. Remember that in between upgrade and append we set properties, so we can't use append to drive the upgrade.

I was hoping that there was a way to adopt the cloned template and upgrade at the same time, but this seems to be impossible. importNode will immediately upgrade because the clone routine is passed the right context, which causes synchronous upgrades of custom elements during the cloning process.
We'll have to wait for the result of the benchmarks. It would be very unfortunate if it causes a significant slowdown, since that would mean we will have to choose between performance and spec compliance.

We're going to investigate this issue more after 1.0. Right now we advise that elements don't change their own light-dom (that's what ShadowRoots are for), but if you do, don't add light-DOM children until connectedCallback().
Later we'll see if not upgrading template instances until after template processing is acceptable perf-wise.


require templates in the Template constructors (#1108)

Description of changes
Backport of #1108. Last of the currently planned changes for 3.3.0.

The Java & SemVer CI failures are expected, but I'm a little confused about the msrv failure since I can't seem to reproduce it locally. Suggestions welcome.

Seems to reproduce for me on this PR's branch with cargo hack check --rust-version --workspace --all-targets --ignore-private. Weirdly, the error is at cedar-policy/src/tests.rs:3440 with
could not find `test_utils` in `cedar_policy_core`

But cargo test --all-features passes with Rust 1.80, so what about 1.76 is causing it to not find that module?

Issue resolved by updating the cedar-policy Cargo.toml file to match the main branch. (The reason I couldn't reproduce before was that I was too lazy to run cargo install cargo-hack and was trying to reproduce manually... my bad.)


use hadolint binary instead of container image

Simpler to use in ci as it excludes any further dependency on needing podman. Also, nested container run doesn't go well.

/cc @djoshy @cgwalters PTAL


Add summary

Add a summary function.
It might help to have a helper to "compile" multiple functions into a single ufunc with tuple output. But before diving too much into this a quick benchmark should be done to make sure it gives some advantage

It's probably a good time to rethink the summary function. I am mostly happy with our current approach, but we should think about how to accommodate at least some of these metrics too (like min_ss) mc-stan.org/posterior/articles/pareto_diagnostics.html. Maybe we could have 3 sets of keywords to mix as we want.
"stats": mean, sd, hdi_3%, hdi_97%
"c_diagnostics": ess_bulk, ess_tail, and r_hat.
"mc_diagnostics": mcse_mean, ess_basic, pareto_khat, min_ss
"c_diagnostics" and ""mc_diagnostics" are note very good names
Also not sure what to include in "mc_diagnostics" I would like something as compact as possible. maybe just mcse_mean, ess_basic, min_ss?

"c_diagnostics" and ""mc_diagnostics" are note very good names

What are these two categories meant to represent?

c_diagnostics is related to convergence diagnostics and mc_diagnostics is realted to montecarlo error.
Both are related, but in a very schematic way I see the first as the diagnostic you check earlier in the workflow and the latter as the ones you check when you are closer to report results.

Got it, makes sense. I like the idea of dividing the diagnostics by workflow stages. But I also like the current approach where the user requests either moments or quantile-based estimates of location and spread (roughly what "focus" does).
RE "mc_diagnostics" I wonder if it makes sense to adjust the columns based on user-provided expectation(s). e.g. user requests a moment, so the moment, MCSE, and Pareto diagnostics for that moment are included. A user requests a quantile, and the quantile and MCSE are included (Pareto diagnostics make no sense here).
Basic ESS is only relevant when the mean is requested, but if MCSE of mean is already provided, I don't think it's particularly useful. In PosteriorStats, we also use an MCSE-derived heuristic to limit the printed precision of the requested estimate.


AJV 6.5.2 is not IE11 compatible as it still depends on uri-js version 4.2.1 instead of 4.2.2

What version of Ajv are you using? Does the issue happen if you use the latest version?
4.5.2 / yes
The problem
Ajv 4.5.2 depends on uri-js version ^4.2.1. This version of uri-js had the problem that it forced packers to use the "esnext" dist instead - as previous versions - the es5 version. This lead into breaking IE11 compatibility. Regarding the uri-js project they fixed the issue with uri-js version 4.2.2. Ajv still uses depends on the broken version 4.2.1 even if they released 4.2.2 some months ago.
So after upgrading our projects to the latest AJV (4.5.2) it crashes in IE 11 because esnext-code of uri-js gets picked up.
See this issue over at uri-js for more information. Especially the last 2-3 comments.
Expected
Expect to get the ES5 code of uri-js which should be true if AJV depends on the 4.2.2 patch of uri-js. If AJV claims to be IE11 / ES5 compatible it should fix this dependency so the workaround below is not required to use AJV with ES5.
Workaround
In the app's package.json one can add a dependency to uri-js version ^4.2.2 which will then force NPM/yarn to use the patched version of uri.js. But this is a workaround as the app itself does not require this library.

@mduu The dependency version is "^4.2.1" that means that the latest "4.x.x" version will be installed - it is currently 4.2.2. Why it is an issue and why #834 is needed?

@epoberezkin Very good question. I'm not really a NPM/yarn expert but Yarn (and npm) both installed 4.2.1 of uri-js instead the latest one. I know that Nuget would do it like you described if it is only the third number (others will not auto-upgrade). When many packages have dependency on uri-js package-manager normally try to negotiate the version that matches best for all. This may be not the latest one but something in between. Not sure about this with NPM/yarn.
As we know that AJV has problems with 4.2.1 I suggest to upgrade its dependency to a version we know should be "safe" for AJV. This way AJV can guarantee it will work properly (even with ES5).

Yarn (and npm) both installed 4.2.1 of uri-js instead the latest one

Does it happen when you install from empty node_modules folder? The problem may be with your npm setup as well.

When many packages have dependency on uri-js package-manager normally try to negotiate the version that matches best for all

Where is this behaviour described? My understanning is that if the latest version is 4.2.2 than it is the only one that matches “^4.2.1”

Does it happen when you install from empty node_modules folder? The problem may be with your npm setup as well.

I got 4.2.1 on a clean node_modules directory.

Where is this behaviour described? My understanding is that if the latest version is 4.2.2 than it is the only one that matches “^4.2.1”

I searched the web quite a while but could not find any documentation how NPM negotiate the final version number.
But thinking of it its relatively clear that it will not just use the latest patch but it needs to check the dependencies of other packages to the same target package. Eg. ajv says ^4.2.1 but another package I have in package.json may say "4.2.1". Then I definitively get 4.2.1 which breaks things. If ajv knows that 4.2.1 results in problem and therefore specify ^4.2.2 then the above situation results in a error the developers have to check and solve manually which is correct.

Just ran into the same issue, beware you need to update ajv-error-messages as well if you're using it, but it hasn't been updated yet...


how to program with pdfminer.six

i have been programming with pdfminer3k because there are some examples in the internet
but for somg reason i need CJK support so i install pdfminer.six
i found there are all command functions
is there any coding sample that programming with pdfminer.six?
attatch my code in pdfminer3k
from pdfminer.pdfparser import PDFParser,PDFDocument
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import PDFPageAggregator
from pdfminer.layout import LTTextBoxHorizontal,LAParams,LTFigure
from pdfminer.pdfinterp import PDFTextExtractionNotAllowed
def get_mulu_text1(pdf_path):
    fp = open(pdf_path, 'rb')

    praser = PDFParser(fp)

    doc = PDFDocument()

    praser.set_document(doc)
    doc.set_parser(praser)

    doc.initialize()

    if not doc.is_extractable:
        raise PDFTextExtractionNotAllowed
    else:
        rsrcmgr = PDFResourceManager()
        laparams = LAParams()
        device = PDFPageAggregator(rsrcmgr, laparams=laparams)
        interpreter = PDFPageInterpreter(rsrcmgr, device)
        for page in doc.get_pages():  # doc.get_pages() 获取page列表
            interpreter.process_page(page)
            layout = device.get_result()
            if (layout.pageid <10):
                for x in layout:
                    if (isinstance(x, LTTextBoxHorizontal)):
                        results = x.get_text()
                        print(results)
                        print('-')
            else:
                break

above codes seems work in pdfminer3k,but not work in pdfminer.six because of some library changed name or other reasons
is there any example for me like above to use pdfminer.six extracting informations with python?
thanks everyone!!

Hi @netbusxp, thanks for your question. Could you share the pdf that you are using with this code? This would make it easy for others to reproduce the issue.
And about the issue: what is it? :smile: you don't mention any specifics about the problem, only that there is one.

@pietermarsman above code is used for pdfminer3k (i have search these days and found that pdfminer3k is a extent library wrtiien by others--also in github)
and here is the pdf
2019-03-16-600508-上海能源：2018年年度报告.PDF
my task is to recongnize the catalogs(in this file in page 3) and extarct information by catalog name(in this file i extract 12 content by catalog name ) saving in a excel,i have to handle 40000+ pdfs like above so i have to program with python with it
above code is worked in pdfminer3k(using pip install pdfminer3k) and can print first 10 pages results in command line,but the question is pdfminer3k is not support CJK ,so i changed to pdfminer.six
but i dont know how to program with six,the above codes is not work in six,because maybe the library name is different
what i need is programming example to handle the pdf i uploaded by python
only print the result in the command line is okay,if it is possible can add some conditions like page number\or print by texts not pages

I've adjusted your example a little and reduced the code a bit. The following example works for me. Does it also work for you?
from pdfminer.converter import PDFPageAggregator
from pdfminer.layout import LAParams
from pdfminer.layout import LTTextBoxHorizontal
from pdfminer.pdfinterp import PDFPageInterpreter
from pdfminer.pdfinterp import PDFResourceManager
from pdfminer.pdfpage import PDFPage

def get_mulu_text1(pdf_path):
    resource_manager = PDFResourceManager()
    page_aggregator = PDFPageAggregator(resource_manager, laparams=LAParams())
    page_interpreter = PDFPageInterpreter(resource_manager, page_aggregator)

    with open(pdf_path, 'rb') as fin:
        for page in PDFPage.get_pages(fin, pagenos=[3]):
            page_interpreter.process_page(page)
            layout = page_aggregator.get_result()
            for element in layout:
                if isinstance(element, LTTextBoxHorizontal):
                    print(element.get_text())

get_mulu_text1('2019-03-16-600508-.2018.PDF')

Note that:

It only extracts text from page 3 because I know that this is the page you are interested in. You can parse all pages by removing pagenos=[3].
The output are all characters that I can't read but I'm pretty sure that it parses the CJK characters correctly.

I'm closing this question because I am considering it answered. Feel free to reopen if you keep having problems.

I've adjusted your example a little and reduced the code a bit. The following example works for me. Does it also work for you?
from pdfminer.converter import PDFPageAggregator
from pdfminer.layout import LAParams
from pdfminer.layout import LTTextBoxHorizontal
from pdfminer.pdfinterp import PDFPageInterpreter
from pdfminer.pdfinterp import PDFResourceManager
from pdfminer.pdfpage import PDFPage

def get_mulu_text1(pdf_path):
    resource_manager = PDFResourceManager()
    page_aggregator = PDFPageAggregator(resource_manager, laparams=LAParams())
    page_interpreter = PDFPageInterpreter(resource_manager, page_aggregator)

    with open(pdf_path, 'rb') as fin:
        for page in PDFPage.get_pages(fin, pagenos=[3]):
            page_interpreter.process_page(page)
            layout = page_aggregator.get_result()
            for element in layout:
                if isinstance(element, LTTextBoxHorizontal):
                    print(element.get_text())

get_mulu_text1('2019-03-16-600508-.2018.PDF')

Note that:

It only extracts text from page 3 because I know that this is the page you are interested in. You can parse all pages by removing pagenos=[3].
The output are all characters that I can't read but I'm pretty sure that it parses the CJK characters correctly.

Yesterday i also finished this question,but your code seems better than me,i am working now,afterwork i will try
thanks very much!!!!!!!! this problem troubles me nearly 20days~
another question:is there any manual or guideline about pdfminer.six
reading the source code semms a little difficult for me

You can checkout the old instructions, the have rebirth on read the docs: https://pdfminer-docs.readthedocs.io/programming.html
Our documentation is still lacking. We hope to fix that soon.

okay thanks i will try

@pietermarsman hi these days when i extract pdf i met a question
009-000682-东方电子：2009年年度报告.PDF
Traceback (most recent call last):
File "D:/upan/pdfminer.six-20181108/tools/trans.py", line 37, in
get_mulu_text1(r'009-000682-东方电子：2009年年度报告.PDF')
File "D:/upan/pdfminer.six-20181108/tools/trans.py", line 25, in get_mulu_text1
for page in PDFPage.get_pages(fin, pagenos=[3]):
File "D:\python\lib\site-packages\pdfminer.six-20181108-py3.7.egg\pdfminer\pdfpage.py", line 129, in get_pages
doc = PDFDocument(parser, password=password, caching=caching)
File "D:\python\lib\site-packages\pdfminer.six-20181108-py3.7.egg\pdfminer\pdfdocument.py", line 566, in init
xref.load(parser)
File "D:\python\lib\site-packages\pdfminer.six-20181108-py3.7.egg\pdfminer\pdfdocument.py", line 195, in load
(_, obj) = parser.nextobject()
File "D:\python\lib\site-packages\pdfminer.six-20181108-py3.7.egg\pdfminer\psparser.py", line 574, in nextobject
(pos, token) = self.nexttoken()
File "D:\python\lib\site-packages\pdfminer.six-20181108-py3.7.egg\pdfminer\psparser.py", line 500, in nexttoken
self.fillbuf()
File "D:\python\lib\site-packages\pdfminer.six-20181108-py3.7.egg\pdfminer\psparser.py", line 230, in fillbuf
raise PSEOF('Unexpected EOF')
pdfminer.psparser.PSEOF: Unexpected EOF
so what is this meaning or what shall i do
may be the form of the file have some problem or i'm lack of some libraries?

It probably means the pdf is broken. Pdfminer.six is robust for some variation in pdf's but it has its limits.
If this also happens to other pdf's or if this pdf is important to you, please create a new issue for it.


workbox-background-sync/Queue replayRequests change order of request`s

Library Affected:
workbox-background-sync
Browser & Platform:
Caught on a Mobile Chrome, but can thrown on any.
Issue or Feature Request Description:
Queue.replayRequests operation don't block queue storage for adding new request. That can throw switch place of first two request.
Screenshots:

For replying issue need in time that replayRequests executed push new request. (In my case thrown by 2 inline request).
Code of my service worker:
import { precacheAndRoute, createHandlerBoundToURL } from 'workbox-precaching';
import { registerRoute, NavigationRoute } from 'workbox-routing';
import { skipWaiting, clientsClaim } from 'workbox-core';
import { CacheFirst } from 'workbox-strategies';
import { CacheableResponsePlugin } from 'workbox-cacheable-response';
import { ExpirationPlugin } from 'workbox-expiration';
import { IgnoreURLParameterPlugin } from './workbox-plugins';
import { Queue } from 'workbox-background-sync';
import * as Messenger from '../src/redux/reducers/util/Messenger';

declare const self: Window & ServiceWorkerGlobalScope & WorkerGlobalScope;

skipWaiting();
clientsClaim();

precacheAndRoute(self.__WB_MANIFEST);

registerRoute(
  ({ url }) => url.origin.includes('s3.amazonaws.com') && !url.pathname.includes('.pdf'),
  new CacheFirst({
    cacheName: 'image-cache',
    plugins: [
      new CacheableResponsePlugin({
        statuses: [200],
      }),
      new IgnoreURLParameterPlugin(),
      new ExpirationPlugin({
        maxEntries: 2000,
        maxAgeSeconds: 3 * 24 * 60 * 60,
      }),
    ],
  }),
  'GET'
);

const handler = createHandlerBoundToURL('/index.html');
const navigationRoute = new NavigationRoute(handler);
registerRoute(navigationRoute);

const queue = new Queue('requestsQueue', {
  maxRetentionTime: 3 * 24 * 60,
  onSync: async (event): Promise<void> => {
    try {
      await event.queue.replayRequests();
    } finally {
      Messenger.sendMessage<number>('requests_queue_length', (await event.queue.getAll()).length);
    }
  },
});

async function sendRequestsQueueLength(): Promise<void> {
  Messenger.sendMessage<number>('requests_queue_length', (await queue.getAll()).length);
}

const replayRequestsQueue = async (): Promise<void> => {
  try {
    await queue.replayRequests();
  } finally {
    await sendRequestsQueueLength();
  }
};

Messenger.createMessageListener<null>('replay_requests_queue', () => {
  replayRequestsQueue();
});

const processRequest = async (request: Request): Promise<Response> => {
  const isQueueEmpty = (await queue.getAll()).length === 0;

  if (isQueueEmpty) {
    try {
      return await fetch(request.clone());
    } catch {} // eslint-disable-line no-empty
  }

  await queue.pushRequest({ request });
  replayRequestsQueue();
  return new Response(null, { status: 200 });
};

self.addEventListener('online', () => {
  replayRequestsQueue();
});

self.addEventListener('fetch', (event: FetchEvent) => {
  if (
    !event.request.url.includes('yembo.ai') ||
    !['POST', 'PUT', 'DELETE'].includes(event.request.method) ||
    event.request.url.endsWith('/login')
  ) {
    return;
  }

  event.respondWith(processRequest(event.request));
});

Short dive in Queue code. I am create indicator in my code for running replayRequest function and this help avoid dangerous behavior.
But question is steel open.
Why only sync event check that sync is in Progress? But not a replayRequest function?
@jeffposnick Can I create PR with this check?

@jeffposnick can you share details for how to check with help of workbox that PWA app on android/IOS phone help to check app is running in background and keep calling notifications to fetch?

Hi there,
Workbox is moving to a new engineering team within Google. As part of this move, we're declaring a partial bug bankruptcy to allow the new team to start fresh. We realize this isn't optimal, but realistically, this is the only way we see it working. For transparency, here're the criteria we applied:

For PRs, we closed everything that doesn't target the current v7 branch.
For Issues, we closed everything that was created before 2024.

Thanks, and we hope for your understanding!
The Workbox team


Bump error-chain version

Shouldn't break anything.

Sorry, I've been travelling for a couple weeks. Thanks for this!


Incorrect usage of Paper's async chunk API

WorldBorder is loading chunks through the forceLoad api, and not the paper api. For example:
https://github.com/Brettflan/WorldBorder/blob/master/src/main/java/com/wimbli/WorldBorder/WorldFillTask.java#L326
This is because, internally, force load api will load the target chunk. A fix is to delay adding the force loaded tag until after the chunk is loaded, which can be done via the thenAccept method on the future returned by paper api.
There's no rush to resolve this issue given over at paper we actually have not implemented asynchronous chunk loading for 1.14, so currently it makes no difference.

Ah, so World.setChunkForceLoaded actually loads the chunk rather than just setting a flag to prevent it from being unloaded? I didn't seen an indication of that in the documentation.
https://hub.spigotmc.org/javadocs/spigot/org/bukkit/World.html#setChunkForceLoaded-int-int-boolean-
Do you know of another way to set that flag without also loading the chunk?

No, the flag must be set after loading the chunk

Since our API returns a completablefuture, you can use the thenAccept method to set the force loaded flag - alternatively the plugin ticket api I added to spigot recently.

I don't have a whole lot of time recently, and I'm also not the one who implemented the Paper API into WorldBorder. I took a look briefly and couldn't see how to do that; could you possibly provide a brief example?

Yes, here's an example:

  	PaperLib.getChunkAtAsync(world, x, z, false).thenAccept((Chunk chunk) -> {
  		if (chunk != null) {
  			// if generated
  			world.setChunkForceLoaded(x, z, true);

  			// alternatively for 1.14.4+
  			//world.addPluginChunkTicket(x, z, pluginInstance);
  		}
  	});

Great, thanks.
Based on that code, with my limited knowledge of CompletableFuture, I'd appreciate if you could check this to be sure it looks correct:
https://github.com/Brettflan/WorldBorder/commit/f88ede397c6625993a6d7412ea74e0ce233875c3

Looks good to me

Cool, thanks again.


Be Able to Run Script in the EMR Terminal

It would be nice if we could run scripts by sshing into EMR, uploading the desired script, and then running into. Because right now, you need to go into JupyterHub and create a terminal in there. Then once created, you'll need to export these variables before you can run the script.

Here's the script that I was trying to run: https://gist.github.com/jbouffard/408d9487c8561e26d7f6ea040c570cdd Ideally, I'd like to have been able to run this script through this command: ./spark-submit ./global_osm_tobler_gps.py. However, I found that I need to include the --jars parameter to make it work, even though it's assinged elsewhere in the GPS code base https://github.com/locationtech-labs/geopyspark/blob/master/geopyspark/init.py#L118

This error
PYSPARK_PYTHON=/usr/bin/python3.4 PYSPARK_DRIVER_PYTHON=/usr/bin/python3.4 spark-submit test.py
...
org.apache.spark.SparkException: Failed to register classes with Kryo
        at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:138)
        at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:289)
        at org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:274)
        at org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:184)
        at org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:269)
        at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:126)
        at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:88)
        at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
        at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:56)
        at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1410)
        at org.apache.spark.scheduler.DAGScheduler.submitMissingTasks(DAGScheduler.scala:997)
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:919)
        at org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:863)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1683)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1675)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1664)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
Caused by: java.lang.ClassNotFoundException: geotrellis.spark.io.kryo.KryoRegistrator
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$5.apply(KryoSerializer.scala:133)
        at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$5.apply(KryoSerializer.scala:133)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
        at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
        at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:133)
        ... 16 more

occurs when attempting to spark-submit this script
from functools import partial
import geopyspark as gps
import fiona
import pyproj

from pyspark import SparkContext
from shapely.geometry import MultiPoint, MultiLineString, shape
from shapely.ops import transform
from geonotebook.wrappers import VectorData, TMSRasterData

# Set up our spark context
conf = gps.geopyspark_conf(appName="San Fran MVP", master="yarn-client")
sc = SparkContext(conf=conf)

list = [1,2,3]
rdd = sc.parallelize(list)
print(rdd.collect())

on EMR using either the hadoop account or an account created by oauth.  The problem is not particular to one or the other, but is actually caused by apparent differences in behavior between spark-submiting a script and running a script through pyspark-shell, which is what Jupyter appears to be doing.  This is not a simple configuration issue and requires deeper investigation.
I will close this issue and open one in the geopyspark repository, because that seems to be the appropriate place.

Similarly, this
aws emr add-steps \
  --cluster-id j-xxxxxxxxxxxx \
  --steps 'Name=Spark,Jar=s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar,Args=[/bin/sh,-c,"GEOPYSPARK_JARS_PATH=/opt/jars PYSPARK_PYTHON=/usr/bin/python3.4 PYSPARK_DRIVER_PYTHON=/usr/bin/python3.4 spark-submit /home/hadoop/test.py"],ActionOnFailure=CONTINUE'

does not work, but this
aws emr add-steps \
  --cluster-id j-xxxxxxxxxxxx \
  --steps 'Name=Spark,Jar=s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar,Args=[/bin/sh,-c,"GEOPYSPARK_JARS_PATH=/opt/jars PYSPARK_PYTHON=/usr/bin/python3.4 PYSPARK_DRIVER_PYTHON=/usr/bin/python3.4 spark-submit --jars /opt/jars/geotrellis-backend-assembly-0.3.1.jar /home/hadoop/test.py"],ActionOnFailure=CONTINUE'

does.


Display time in MM:SS format

Issue Details: https://github.com/olimorris/tmux-pomodoro-plus/issues/13

Before
After

Done ✅

[x] Time comparison to show Pomodoro state (timer/break timer/done) is based on seconds & not minutes
[x] Introduced a new function format_seconds which is used to format the minutes to the appropriate MM:SS format
[x] Introduce a new option for the user to enable seconds & format_seconds to handle response based on this optionset -g @pomodoro_granularity_seconds 'on'    # Displays the time left in `MM:SS` format

[x] ~Figure out where to put the refresh rate of status bar every second tmux set status-interval 1 - Also figure if there is a better way to achieve this when building Tmux plugins~ - Couldn't find any, other than setting the status interval.

How to test these changes?

In your tmux config, point to this branch & enable seconds option:set -g @plugin 'PezCoder/tmux-pomodoro-plus#issues/13/show-seconds'

set -g @pomodoro_granularity_seconds 'on'    # Displays the time left in `MM:SS` format

set -g status-interval 1 # Makes sure tmux updates the status bar every second

Remove the original version from TPM:rm -rf ~/.tmux/plugins/tmux-pomodoro-plus

Trigger TPM install prefix + I
Reload tmux config (or restart tmux)
Optional: You can set the option to configure pomodoro timer to be 1 minutes - to test break scenarios

About updating the status line every second
Chat gpt way - using while infinite loop

❌ Doesn't seem like a great idea, as after adding the loop completely froze my tmux
Set status interval approach - @olimorris
tmux set status-interval 1

The approach you suggested seems to be working alright for me, without any sort of lag in the tmux UI.
This would mean that tmux will update the status bar every second, which is what we are looking for.
❓ I wanted to ask, where should this command be put, should we ask the users to put it in their tmux.conf - as I'm not sure if it would be wise for a plugin to globally set the status update?

❓ I wanted to ask, where should this command be put, should we ask the users to put it in their tmux.conf - as I'm not sure if it would be wise for a plugin to globally set the status update?

I think this is probably the best way forward. I suspect that for some user's configs this could result in some lagging.
I can add a section to the readme explaining how to get additional granularity in the pomodoros: enabling the option and setting tmux set status-interval 1 in their config.

Updated readme & the option as per your suggestions, can you please take a look @olimorris ? - Let me know if any suggestions.

I've just tweaked the readme a little and fixed some of the sh warnings. Can you have a final play, check I've not messed anything up and then I think we're good to go.
Really excited by this and the 1 second interval refresh has seemingly no negative performance impact for myself.

Nice, thank you for the changes!
I just did sanity of the Pomodoro along with the break (by setting the duration to 1), for both "on" & "off" values of pomodoro_granularity_seconds
All good ✅


Clarify x-ms-dynamics-organization header

The x-ms-dynamics-organization header contains the full domain name of the instance that the webhook is triggered from, not the name of the associated tenant

@MarkMpn
Thanks for pointing this out. You are correct.
Sorry it has taken so long, but I have updated the topic directly with another change, so these changes aren't needed now.
The updated topic should appear in the public docs this evening.


mutilGPU

it set net = torch.nn.DataParallel(model,device_ids=[0,1,2,3]).cuda(),but only one gpu work.

Can the original code complete multi GPU training?

where is original code? This code is not original?
------------------ 原始邮件 ------------------
发件人:                                                                                "bubbliiiing/pspnet-pytorch"                                                                                @.>;
发送时间: 2021年6月22日(星期二) 下午4:00
@.>;
<EMAIL_ADDRESS>主题: Re: [bubbliiiing/pspnet-pytorch] mutilGPU   (#11)
Can the original code complete multi GPU training?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

U don't need to set net = torch.nn.DataParallel(model,device_ids=[0,1,2,3]).cuda().
I have set torch.nn.DataParallel(model)

set torch.nn.DataParallel(model)，only one gpu work. If set  torch.nn.DataParallel(model,device_ids=[0,1,2,3]) ,only one gpu work too.
------------------ 原始邮件 ------------------
发件人:                                                                                "bubbliiiing/pspnet-pytorch"                                                                                @.>;
发送时间: 2021年6月23日(星期三) 上午9:28
@.>;
<EMAIL_ADDRESS>主题: Re: [bubbliiiing/pspnet-pytorch] mutilGPU   (#11)
U don't need to set net = torch.nn.DataParallel(model,device_ids=[0,1,2,3]).cuda().
I have set torch.nn.DataParallel(model)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

The code can modify like this.The original code corresponds to error.
------------------ 原始邮件 ------------------
发件人:                                                                                "bubbliiiing/pspnet-pytorch"                                                                                @.>;
发送时间: 2021年6月23日(星期三) 上午9:28
@.>;
<EMAIL_ADDRESS>主题: Re: [bubbliiiing/pspnet-pytorch] mutilGPU   (#11)
U don't need to set net = torch.nn.DataParallel(model,device_ids=[0,1,2,3]).cuda().
I have set torch.nn.DataParallel(model)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Sorry, I'm too busy recently. Have you solved it? I still don't understand why you didn't use multi GPUs.

I had solved  it . I  is using multi GPUs.
------------------ 原始邮件 ------------------
发件人:                                                                                "bubbliiiing/pspnet-pytorch"                                                                                @.>;
发送时间: 2021年7月4日(星期天) 晚上9:44
@.>;
<EMAIL_ADDRESS>主题: Re: [bubbliiiing/pspnet-pytorch] mutilGPU   (#11)
Sorry, I'm too busy recently. Have you solved it? I still don't understand why you didn't use multi GPUs.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.


Timeseries templates

Add convolutional neural network template for timeseries.
Add dsbox logistic regresion template for timeseries.

Codecov Report

Merging #37 into master will increase coverage by 0.09%.
The diff coverage is 66.66%.

@@            Coverage Diff            @@
##           master     #37      +/-   ##
=========================================
+ Coverage   39.21%   39.3%   +0.09%
=========================================
  Files           9       9
  Lines        1321    1323       +2
=========================================
+ Hits          518     520       +2
  Misses        803     803

Impacted Files
Coverage Δ

ta2/search.py
41.15% <66.66%> (+0.36%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e7ae2f...a7440a6. Read the comment docs.


release for es 6.3.0

@vvanholl

This change is

@vvanholl ping?

careful.  They now require plugins to not have /elasticsearch in the zip file.

I built this, but had to repackage the zip so that /elasticsearch/foo.jar became /foo.jar

https://github.com/elastic/elasticsearch/pull/28589


Alert developers on macOS when the app lacks permissions

On macOS, accessibility permissions are required before this library will function at all. See here. This is already documented in the README, but to make it easier on developers using this crate it would be nice if a message could be logged when the app doesn't have permissions. The API could also do this check and expose it as a function somewhere, so that consumers of the crate can write it to their own popup or messaging for their users.
Here's the example from the README, modified to show an example API:
let device_state: DeviceState = match DeviceState::new() {
    Ok(device) => device,
    Err(DeviceError::InsufficientPermissions) => {
            message_box("Please enable Accessibility Permissions");
            return;
    },
    Err(e) => panic!("{}", e),
};

// The rest of the example is unchanged
let mouse: MouseState = device_state.get_mouse();
println!("Current Mouse Coordinates: {:?}", mouse.coords);
let keys: Vec<Keycode> = device_state.get_keys();
println!("Is A pressed? {}", keys.contains(Keycode::A));

I have no idea if Apple's APIs will allow this or not, so this is just spit balling to show usage. Once we know this can work, we can iterate on what a useful API would actually look like.
This commit in dteoh/SlowQuitApps added a popup dialog and is probably a good starting place to figure out how to include this kind of check.

Reading more, I think this is actually a really short function to write:
BOOL hasAccessibility() {
    NSDictionary *options = @{(__bridge id)kAXTrustedCheckOptionPrompt: @YES};
    return AXIsProcessTrustedWithOptions((__bridge CFDictionaryRef)options);
}

Just need to call into AXIsProcessTrustedWithOptions from Rust with an NSDictionary. 🤷♀