Expand size-limit workflow to run basic version on forks

Description

Runs a simpler version of our size-limit check for PRs from forks
We can’t run the full-blown size-limit action from forks because it needs permission to comment on PRs, which is impossible to permit safely from forks. Instead, we run the basic size-limit CLI just for the PR branch. This won’t create a PR comment, but will at least fail for changes that breach the current budget.

Updates our file-size budgets after bump in size caused by #742

size-limit report 📦

Path
Size

/index.html
5.12 KB (0%)

/_astro/*.js
20.11 KB (0%)

/_astro/*.css
13 KB (0%)


Troubles trying to record the user voice

Hi Dave,
I noticed a typo at line 209 in conference.js<EMAIL_ADDRESS>)
- assert.equals(typeof file === 'string', '\'file\' parameter must be provided') ;
+ assert.ok(typeof file === 'string', '\'file\' parameter must be provided') ;
Now, I'm having troubles trying to record the user voice for an IVR test app (with realtime audio analysis on a separate node). I'm using the recording functionalities from the drachtio-fmsrf Conference Class and the latest docker-drachtio-freeswitch-mrf image.
Is this the correct way or do you think there is a better way to record/stream the user voice (like a Freeswitch api call) ?
The problem I've comes using this fragment of code :
req.app.locals.ms.createConference('testconf',
    (err, conf) => {
        if ( err ) {
            console.log(`error createConference: ${JSON.stringify(err)}`) ;
	    throw err ;
	}
	console.log(`conference created: ${JSON.stringify(conf)}`) ;

        // >>> local path on host, not on the docker image <<<
	conf.startRecording('/home/massimo/drachtio/recordings/record.wav',
	    function(err, results){
	        if ( err ) {
		    console.log(`error startRecording: ${JSON.stringify(err)}`) ;
		    throw err ;
		}
		console.log(`recording results: ${JSON.stringify(results)}`) ;
    	    }
	);

    }
);

and the test app throw an error showing an "-ERR record Command not found!\n" :
connected to media server
connected to drachtio listening on tcp/<IP_ADDRESS>:5060,tcp/<IP_ADDRESS>:5060,tcp/<IP_ADDRESS>:5060,tcp/[::1]:5060,udp/<IP_ADDRESS>:5060,udp/<IP_ADDRESS>:5060,udp/<IP_ADDRESS>:5060,udp/[::1]:5060

received an authenticated registration request:  { type: 'register',
  expires: 900,
  contact:
   [ { name: undefined,
       uri: 'sip:1001@<IP_ADDRESS>:56169;ob',
       params: {} } ],
  aor<EMAIL_ADDRESS>}

conference created: { "name":"testconf","state":1,"memberId":-1,"confConn":{"confName":"testconf","createArgs":{"flags":["endconf","mute","vmute"]},"endpointUuid":"62f8121a-30dc-11e7-9217-ffa3ff9f17a5"},"maxMembers":-1,"locked":false}

error startRecording:
{ "headers":
   [ { "name":"Content-Type","value":"api/response" },
     { "name":"Content-Length","value":31 }
   ],
   "hPtr":null,
   "body":"-ERR record Command not found!\n" }

/home/massimo/drachtio/rectest/rectest.js:66
			throw err ;
			^

Sorry, looks like I did not implement those commands correctly.  Can you update to the latest drachtio-fsmrf (0.102.8) and try again?  Thanks for the correction on the typo, I included that as well.

I've added a simple conference record example, along the lines of what you are doing:
https://github.com/davehorton/drachtio-example-conference-record
Glad to hear you are using the docker image!  Let me know if you have any issues, or questions on that.  I have docker images for the drachtio-server as well, are you also using that?

Definitely I don't know how to thank you for your prompt response (and coding too).
The simple record example is working fine and the recording quality as well is excellent.
connected to drachtio listening on tcp/<IP_ADDRESS>:5060,tcp/<IP_ADDRESS>:5060,tcp/<IP_ADDRESS>:5060,udp/<IP_ADDRESS>:5060,udp/<IP_ADDRESS>:5060,udp/<IP_ADDRESS>:5060
conference established...
received an authenticated registration request:  { type: 'register',
  expires: 900,
  contact:
   [ { name: undefined,
       uri: 'sip:1001@<IP_ADDRESS>:36185;ob',
       params: {} } ],
  aor<EMAIL_ADDRESS>}
joined caller to conference
caller removed from conference
received an authenticated registration request:  { type: 'unregister',
  expires: 0,
  contact:
   [ { name: undefined,
       uri: 'sip:1001@<IP_ADDRESS>:36185;ob',
       params: {} } ],
  aor<EMAIL_ADDRESS>}

Docker is a great solution for standardized software development and i'm thinking about using the docker image for the drachtio-server too.
I'm working on a speech recognition system but for fast (IVR) response I need to analyze the audio stream in realtime for an accurate silence detection.
Thank you once again, Dave.

Just as FYI, using Docker I run a drachtio-server like this:
$ docker run -d --name drachtio --net=host \
drachtio/drachtio-server drachtio --contact "sip:*;5060;transport=udp"

and a freeswitch like this:
$ docker run -d --name freeswitch --net=host \
-v /home/deploy/docker-data/freeswitch/log:/usr/local/freeswitch/log \
-v /home/deploy/docker-data/freeswitch/sounds:/usr/local/freeswitch/sounds \
-v /home/deploy/docker-data/freeswitch/recordings:/usr/local/freeswitch/recordings \
drachtio/drachtio-freeswitch-mrf freeswitch \
--sip-port 5080 --tls-port 5081 --rtp-range-start 20000 --rtp-range-end 20100

I can then run my nodejs outside of a container, or I can run it in a container by creating a Dockerfile and using https://hub.docker.com/r/drachtio/nodejs
$ docker build -t conference-record .
$ docker run -ti --net=host --name conference-record --rm conference-record

The nice part of running drachtio-server and freeswitch in containers is then I can easily docker stop drachtio and docker start drachtio at any time I need it, and same with freeswitch

Yes, it's absolutely a useful and clean way to develop a complex architecture.
There is another advantage I'll try soon, Docker on Linux supports shared UNIX-domain sockets and named pipes for IPC (and this is really cool).


[Maestro version, e.g. v1.2.3]

Good afternoon
I can’t get to the required element in a horizontal block.
The first step is to move to this block
- scrollUntilVisible:
    centerElement: true
    element:
      id: MagazineHorizontalList
    direction: DOWN

Next I try to go to the element in this block
- scrollUntilVisible:
  centerElement: true
  direction: right
  element:
    id: MagazineListThirdItem

As a result, the scroll does not work.
Construction type is not supported.- scrollUntilVisible:
direction: right
container:
  id: "horizontal-list-container"
element:
  id: "item-5"

How to solve this problem?

Hey, this issue is not reproducible, therefore there's nothing we can do but close it. Please open a new one with more info.


Connect api profile reuse form

Related to PR #239
Changes

re-use Organization APIs form
restructure some code
cleanup
lint

@brylie Reviewing

@brylie

Specify right label and help text for modal form

These is showing incorrect message when modal form is closed

Specify right label and help text for modal form
I updated the label localization in the i18n file. We probably don't need a help text for this form, but feel free to add it in your PR.

These are showing an incorrect message when modal form is closed. Although the meaning of the message is correct.

I made the success message more generic, so it can apply in both scenarios.

I think it's fine and is ready to merging

Cool beans!

Go ahead and merge, if you approve the changes.


Polyfill Suggestion

Hi,
I found a small bit of JS and reworked it to only load the w3c polyfill if the browser doesn't yet support the IntersectionObserver API, you might want to use it in your FAQ.  If there's a more efficient way of doing this so that you get the native IntersectionObserver behavior on Chrome/Firefox/Edge and the polyfill on Safari/IE do let me know.
if (!('IntersectionObserver' in window)) {
var js = document.createElement("script");
js.src = "https://raw.githubusercontent.com/w3c/IntersectionObserver/master/polyfill/intersection-observer.js";
document.getElementsByTagName('head')[0].appendChild(js);
}

@TiKevin83 thanks for sharing this, have updated FAQs


Value ignored for a boolean as a selectfieldset

Here is the simplest, clear example:
{   "schema": {     "field": {       "type": "boolean",       "title": "A field"     }   },   "form": [     {       "key": "field",       "type": "selectfieldset",       "items": [         {"type":"help", "helpvalue": "0"},         {"type": "help", "helpvalue": "1"}       ]     },     {       "type": "submit",       "title": "Submit"     }   ],   "value": {     "field": false   } }
When I put this into the playground, all behavior is that of "field" being true.

Hello
I tested that right now in the playground and when I change the option, I get "true" or "false" depending on the selection, so it seems to be working ? Or maybe I didn't get what your problem is.
I'm using your schema (prettified) :
{
  "form": [
    {
      "items": [
        {
          "helpvalue": "0",
          "type": "help"
        },
        {
          "helpvalue": "1",
          "type": "help"
        }
      ],
      "key": "field",
      "type": "selectfieldset"
    },
    {
      "title": "Submit",
      "type": "submit"
    }
  ],
  "schema": {
    "field": {
      "title": "A field",
      "type": "boolean"
    }
  },
  "value": {
    "field": false
  }
}

Closing as there is no update from the OP. Please reopen if this is still an issue on your side


Voice recognition often stops prematurely

Willow will often stop recognizing mid-sentence, especially if I pause slightly to find my next word. It's very trigger-happy with stopping, to the extent where I think about what I want to say in its entirety before I talk to Willow.

Have you tried increasing the VAD timeout? By default it is 300ms, meaning if you stop talking for 300ms it assumes you are done with your command. You can raise this higher which would give you more time before it stops listening. 🙂

Ah, I didn't realize this was an option, thank you. Perhaps it can be documented better, in a troubleshooting FAQ or something similar? Or is it already and I just made a fool of myself? 😛


Plutus Relese <IP_ADDRESS> with plcVersion110 causing AlwaysPassValidator to fail

Summary
I recently updated my project to plutus tag-<IP_ADDRESS> for a minting contract, the validator is:
{-# INLINEABLE mkValidator #-}
mkValidator :: MintScriptParameter -> MintRedeemer -> ScriptContext -> Bool
mkValidator parameter redeemer ctx = case redeemer of
  FirstMint -> True
  FutureMint -> True

{-# INLINEABLE mkWrappedValidator #-}
mkWrappedValidator :: MintScriptParameter -> BuiltinData -> BuiltinData -> ()
mkWrappedValidator parameter redeemer context =
  check $
    mkValidator
      parameter
      (unsafeFromBuiltinData redeemer)
      (unsafeFromBuiltinData context)

mintTokenValidator parameter =
          $$(PlutusTx.compile [||mkWrappedValidator||])
            `applyCode` PlutusTx.liftCode plcVersion110 parameter

but while compiling the transaction, i get
{"message":"","type":"PlutusScriptError"}
trying to resolve this by using plcVersion100 in all the smart contract validators,  i get the following error:
Cannot apply two programs together: the first program has version Version {_versionMajor = 1, _versionMinor = 1, _versionPatch = 0} but the second program has version Version {_versionMajor = 1, _versionMinor = 0, _versionPatch = 0}
CallStack (from HasCallStack):
  error, called at prelude/PlutusPrelude.hs:250:29 in plutus-core-<IP_ADDRESS>-558ad5ec54ffdc1365ea689f545446419a80d20ff2991118785bc2d8240605e1:PlutusPrelude

Steps to reproduce the behavior

Try out this example:

data MintScriptParameter = MintScriptParameter
  { proposalTokenName :: TokenName,
    quorumNFTName :: TokenName,
    txOutRef :: TxOutRef
  }
  deriving (Generic, Prelude.Show)
PlutusTx.makeIsDataIndexed 'MintScriptParameter [('MintScriptParameter, 0)]
PlutusTx.makeLift ''MintScriptParameter

data MintRedeemer = FirstMint | FutureMint

PlutusTx.makeIsDataIndexed
  ''MintRedeemer
  [ ('FirstMint, 0),
    ('FutureMint, 1)
  ]

{-# INLINEABLE mkValidator #-}
mkValidator :: MintScriptParameter -> MintRedeemer -> ScriptContext -> Bool
mkValidator parameter redeemer ctx = case redeemer of
  FirstMint -> True  -- setting redeemer cases to True for "AlwaysPass"
  FutureMint -> True

{-# INLINEABLE mkWrappedValidator #-}
mkWrappedValidator :: MintScriptParameter -> BuiltinData -> BuiltinData -> ()
mkWrappedValidator parameter redeemer context =
  check $
    mkValidator
      parameter
      (unsafeFromBuiltinData redeemer)
      (unsafeFromBuiltinData context)

mintTokenValidator parameter =
          $$(PlutusTx.compile [||mkWrappedValidator||])
            `applyCode` PlutusTx.liftCode plcVersion100 parameter

Use the following function to generate the cbor as ByteString:

mintingScript a = case mintTokenValidator a of
  Left _ -> error "Error Parsing MintingScript"
  Right compiledCode -> BS8.unpack $ parseToScript compiledCode
  where
    parseToScript :: CompiledCode a -> BS8.ByteString
    parseToScript compiledCode = Base16.encode $ fromShort $ serialiseCompiledCode compiledCode

compile the minting transaction

{
    "mint": [
        {
            "script": {
                "cborHex": "58910101003222328001919192999ab9a3370e90000010c00054ccd5cd19b874800800860042c6aae78008d55ce8009baa002335122333335122002222800801c00a0022129002800922010850726f706f73616c0048810651756f72756d0033002489200d967444377dd05f4a5e9d9bfdf168b959bed43655635f8daa63a07f1bd2ff7c00480108a002005001109480140041",
                "description": "",
                "type": "PlutusScriptV2"
            },
            "redeemer": {
                "constructor": 0,
                "fields": []
            },
            "amount": {
                "Token1": 1,
                "Token2": 1
            }
        }
    ],
    "outputs": [
        {
            "address": "addr_test1wq3zhud8la6u6ygmpu30at3vt8k3juathxxjujhklzapxtsj3lx7f",
            "value": "934d8750c215e0a05232c34cadf5dac5825d78f0820dd5178ab784f4.Token1",
            "datum": {
                "constructor": 0,
                "fields": []
            }
        },
        {
            "address": "addr_test1wq3zhud8la6u6ygmpu30at3vt8k3juathxxjujhklzapxtsj3lx7f",
            "value": "934d8750c215e0a05232c34cadf5dac5825d78f0820dd5178ab784f4.Token2",
            "datum": {
                "constructor": 0,
                "fields": []
            }
        }
    ]
}

Actual Result
The transaction generates a ScriptError:
{"message":"","type":"PlutusScriptError"}
Expected Result
The transaction should compile with no errors as the minting policy is generated in such a way that no-matter what the redeemer is, the validation will always be True.
Describe the approach you would take to fix this
No response
System info
Plutus Tag : <IP_ADDRESS>
Cabal.Project uses:
-- Custom repository for cardano haskell packages, see CONTRIBUTING for more
repository cardano-haskell-packages
  url: https://chap.intersectmbo.org/
  secure: True
  root-keys:
    3e0cce471cf09815f930210f7827266fd09045445d65923e6d0238a6cd15126f
    443abb7fb497a134c343faf52f0b659bd7999bc06b7f63fa76dc99d631f9bea1
    a86a1f6ce86c449c46666bda44268677abf29b5b2d2eb5ec7af903ec2f117a82
    bcec67e8e99cabfa7764d75ad9b158d72bfacf70ca1d0ec8bc6b4406d1bf8413
    c00aae8461a256275598500ea0e187588c35a5d5d7454fb57eac18d9edb86a56
    d4a35cd3121aa00d18544bb0ac01c3e1691d618f462c46129271bccf39f7e8ee

-- See CONTRIBUTING for information about these, including some Nix commands
-- you need to run if you change them
index-state:
  , hackage.haskell.org 2024-03-20T14:55:30Z
  , cardano-haskell-packages 2024-03-20T14:22:58Z

... *Package Descriptions*

-- You never, ever, want this.
write-ghc-environment-files: never

-- Always build tests and benchmarks.
tests: True
benchmarks: False

source-repository-package
  type: git
  location: https://github.com/dquadrant/kuber
  tag: 0bfea4fea31eb20ce9fbc33cfe9907f92a11e5f1

constraints:
    hedgehog >= 1.0
  , bimap >= 0.4.0
  , libsystemd-journal >= 1.4.4
  , systemd >= 2.3.0
    -- systemd-2.3.0 requires at least network <IP_ADDRESS> but it doesn't declare
    -- that dependency
  , network >= <IP_ADDRESS>
  , HSOpenSSL >= <IP_ADDRESS>

package snap-server
  flags: +openssl

package comonad
  flags: -test-doctests

allow-newer:
  *:aeson,
  monoidal-containers:aeson,
  size-based:template-haskell

This error, {"message":"","type":"PlutusScriptError"} - where does it come from?
Do you have any additional information about the error, e.g. logs or stack traces?

i get that while compiling the transaction json fom kuberIDE

The error is produced while calculating execution units from Cardano-API

There isn't enough information in the error as it is presented in this ticket:
Cardano API's error embeds Plutus.EvaluationError together with additional textual information:
ScriptErrorEvaluationFailed Plutus.EvaluationError [Text.Text]

I assume this information is lost somewhere between Cardano API and Kuber.
Without seeing the Plutus.EvaluationError and without knowing the used plutus-ledger-api version its impossible for Plutus developers to reason about the problem.
Please contact maintainers of these repositories for the details.

We logged the error:
{"message":"CodecError Deserialization : PlutusCoreLanguageNotAvailableError {sdeAffectedVersion = Version {_versionMajor = 1, _versionMinor = 1, _versionPatch = 0}, sdeThisPv = 9} : []","type":"PlutusScriptError"}

One potential reason you are getting this error is that you are using either Plutus Language Version V1 or V2, but the Plutus Core version 1.1.0 is only supported by the V3.
(Different versions are explained here)

@reeshavacharya please report back if you have fixed the problem so that we can close this issue. Thank you!

@Unisay i have not fixed it yet. but this issue can be closed.


Model's predictive performance depends upon system memory.

System information.

Have I written custom code (as opposed to using a stock example script provided in Keras): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 2.6.0
Python version: 3.7.12
Bazel version (if compiling from source):
GPU model and memory: Tesla P100-PCIE-16GB
Exact command to reproduce:

Describe the problem.
Me and my colleague (@sayakpaul) are training the same composite neural network that we have written but we are observing major discrepancies in the train, validation and test performance (in terms of evaluation metrics). To ensure consistency, we are pulling the code from the same branch, using the same commit id and are not making any local changes. We are both using Google Colab Pro with a P100 GPU.
Digging deeper we realised that one of us was using a High RAM instance for the Colab notebook which resulted in giving much better results. The other person could replicate the result by switching to a High RAM instance. This was quite confusing as we were even using the same batch size.
Furthermore we investigated the effect of RAM and GPU and found that this behaviour is independent of GPU but very much dependant on RAM. A High RAM instance of Google Colab offers 25GB of RAM and apparently, the model converges nicely when the RAM is the same range (25-26 GB). We spun up a GCP Vertex AI Notebook with 30GB of RAM and tried training the model, but the performance was sub-par again. Then we tried the same with a ~26GB instance and the performance was back to being decent. Finally we reduced the RAM to 13 GB and got worse results as before.
The model comprises of the following components:

A pretrained BERT model from Tensorflow Hub (we are not updating the weights for this).
A GRU layer.
2 IntegerLookup layers.
2 Dense layers.

We read the data from TFRecords for training, validation and testing.
Describe the current behavior.
The neural network gives the best prediction performance (accuracy/precision) when trained in an instance with 25-26 GB of RAM.
Describe the expected behavior.
The size of RAM should not affect the validation and test performance of a model.

@Nilabhra , any update with the code?

@sachinprasadhs Hi! Unfortunately we didn't get the time to make the code available. It's going to be a bit of a hassle to do that.
We did realise that the memory might not have been the actual factor here, but rather the CPU cores are. It's a combination of CPU cores and the behaviour of tf.data.Dataset.interleave.

Is your model's predictive performance sensitive to the ordering of the input datasets?

Is your model's predictive performance sensitive to the ordering of the input datasets?

We are realizing that it was the number of CPU cores rather than system memory that led to the discrepancy. Disabling interleaving of the tfrecords changes the performance drastically in any setting.
I'm closing this issue because the discrepancy is gone after using more data for training.


Update circleci orb

This is required for 2.32.0.
Signed-off-by: Julien Pivotto<EMAIL_ADDRESS>

:+1: Thanks!


Test EDN in Auto Mode

This PR solves #7424. It contains the following components:

EDN DIF: Implement the functionality of dif_edn_set_auto_mode() and add a function to check if EDN is idle.
Chip-level test involving OTBN and EDN: Re-initialize EDN in auto mode before OTBN execution.

one of @engdoreis tests actually needs to make use of this.
I hacked up dif_edn a bit for this and added some things to testutil, but it would be good to have it more formalized in this PR.
See #12966

This is now ready for review and, considering tests are passing, should also be ready for first usage. I hope this unblocks your work, @tjaychen and @engdoreis. Sorry for the delay, this got backlogged for me.
I'll be out of office for a week now but I'm happy to implement feedback afterwards.

@engdoreis @andreaskurth could you guys rebase this one and try it out with the test with @engdoreis recently merged?

@engdoreis @andreaskurth could you guys rebase this one and try it out with the test with @engdoreis recently merged?

I have rebased, resolving the conflicts due to the changes that had been made recently in https://github.com/lowRISC/opentitan/pull/12966. I was able to get both entropy_src_edn_reqs_test and otbn_randomness_test to pass (tested in Verilator) by partially reverting the changes of #12966 in the last two (WIP) commits. I think we should discuss the relation of entropy_testutils_auto_mode_init() and entropy_testutils_boot_mode_init() (especially if auto_mode can be used without boot_mode and other assumptions).

@engdoreis @andreaskurth could you guys rebase this one and try it out with the test with @engdoreis recently merged?

I have rebased, resolving the conflicts due to the changes that had been made recently in #12966. I was able to get both entropy_src_edn_reqs_test and otbn_randomness_test to pass (tested in Verilator) by partially reverting the changes of #12966 in the last two (WIP) commits. I think we should discuss the relation of entropy_testutils_auto_mode_init() and entropy_testutils_boot_mode_init() (especially if auto_mode can be used without boot_mode and other assumptions).

@timothytrippel has commented here that he would clean up the entropy src test utils. So I think he's input would be very valuable for that discussion.

@timothytrippel has commented here that he would clean up the entropy src test utils. So I think he's input would be very valuable for that discussion.

Great, thanks! Considering @timothytrippel has approved this PR and all tests pass, I suggest to merge this and then do the clean-up.


Update readme

Looks good! Are you removing most of these as now the website is available for this kind of info?

Yes, less to maintain. Have a basic README, and then most docs in the site.


fix(applications-service-api): fix issue updating interestedParty

Describe your changes
Fix for bug introduced by https://github.com/Planning-Inspectorate/applications-service/pull/898 where the wrong field (should be ID not id) was being passed to the update query which sets the emailed field in the interested party table in the NI DB (to indicate an email confirmation has been sent to the interested party). Added a unit test which was missed from aforementioned PR that would have caught this
Also added the async route wrapper to allow exceptions to be caught and returned as a HTTP response, rather than crashing the whole node process.
Type of change 🧩

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update
[ ] Other (please explain in the description section above)

Checklist before requesting a review

[x] I have performed a self-review of my own code
[x] I have double checked this work does not include any hardcoded secrets or passwords
[ ] I have made corresponding changes to the documentation
[ ] I have provided details on how I have tested my code
[ ] I have referenced the ticket number above
[ ] I have provided a description of the ticket
[x] I have included unit tests to cover any testable code changes

:tada: This PR is included in version 1.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.15.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


!examples partial_string does not work

Underscores has the bot check for exact matches, but I worked in fuzzy searching in 08c01ec.


dataset processed

Could you please explain how you downsample the NGSIM dataset? The original framerate of NGSIM is 10Hz. How do you process the dataset to reduce it to 5Hz while ensuring that the frame ID remains continuous?

The preprocessing of NGSIM is done as discussed in https://arxiv.org/pdf/1805.06771.pdf. This was done for maintaining  consistency in comparing the results of previous works.

The preprocessing of NGSIM is done as discussed in https://arxiv.org/pdf/1805.06771.pdf. This was done for maintaining  consistency in comparing the results of previous works.

The preprocessing of NGSIM is done as discussed in https://arxiv.org/pdf/1805.06771.pdf. This was done for maintaining  consistency in comparing the results of previous works. You can find the preprocessing code here (https://github.com/nachiket92/conv-social-pooling).

However, we found that the data preprocessing code you mentioned does not contain the data downsampling operation, so how do you downsample the data? Because the sampling frequency of your data can be fatal to your results.

Could you please explain how you downsample the NGSIM dataset? The original framerate of NGSIM is 10Hz. How do you process the dataset to reduce it to 5Hz while ensuring that the frame ID remains continuous?

Are you puzzled by the small results presented in the paper?

We will look into it and get back to you.

We will look into it and get back to you.

What are the results of your investigation? I am looking forward to your reply.


Two instances that give exactly the same string aren't the same!

a = moment().set('hour', 2).set('minute', 0).set('second', 0);
b = moment('020000', 'HHmmss');
a.format() === b.format();  //true
a.isSame(b);  //false

A colleague pointed out that isSame() compares the dates up to milliseconds when called without an extra parameter. (https://github.com/moment/moment/blob/develop/moment.js#L2438)
So I did a little test:
a.get('ms');   //123 or some other millisecond at the time 'a' was created
b.get('ms');   //0

But nevertheless, this is the wrong behavior because:
When b is created, it takes the current year, month and day to fill in the missing bits. So it would be unexpected if it doesn't take the milliseconds. Milliseconds are missing from the provided pattern. No big "S" in "HHmmss".

I disagree with your conclusion.
Date is always fetched, but time is never set when you create from string with "time" precision:
If you create with b = moment('02', 'HH'); minutes and seconds are set to 0.
Creating with b = moment('02', 'ss'); leaves hours and minutes set to 0.
Why should only milliseconds be fetched?
In regards to above example:
a = moment().set('hour', 2).set('minute', 0).set('second', 0);
b = moment('020000', 'HHmmss');
a.isSame(b,'minute');  //false
a.isSame(b,'second');  //true
a.isSame(b,'mili');  //false -- same comparison as a.isSame(b);

And (as provided by your edit)
a = moment().set('hour', 2).set('minute', 0).set('second', 0).set('millisecond', 0);
b = moment('020000', 'HHmmss');
a.format() === b.format();  //true
a.isSame(b);  //true

ok. fair enough. I agree with that logic. Closing it.


Support for nullable Enum-Properties

When comparing an object with a nullable enum property and set CompareChildren to false differences in these properties are not detected.
The TypeHelper.CanHaveChildren() - Method returns true.
I think the method IsSimpleType() should handle nullable enums as well

Thanks for reporting this defect.  It will be a while until I can get to this.

Let me know if I can support you in some way.

If you would like to create a PR with a test to fix it that would speed up the process.

Ok, I've created a PR.

Merged thanks so much!

Thank you!! :)

Just one question: Do you also publish a new nuget package?

There is one more Issue remaining before I publish

Released in v4.65


Display Cloud banner on self-hosted instances

Pull Request
Related issue
Fixes #563
What does this PR do?

Display Cloud banner by default
Hide Cloud banner only if query parameter has cloud_banner=false
Update copy

It's not related to this PR, but scrolling makes the banner disappear 🤔
I think that's not a classical behaviour for a banner and this is something we might want to fix.
@Strift feel free to fix it here if that's an easy one, or to open an issue instead (could be a good first issue for contributors too 👍 ).
https://github.com/user-attachments/assets/000d7fe4-92e8-4e09-a366-e36080c1428d
Otherwise, that's looking good to me 🙌

hey @mdubus, thanks for the review. As request I:

made the banner sticky again (sorry for the oversight)
renamed the util file to include Cloud to make things clearer

Thank you @Strift  🙏
But now if I close it, it breaks the UI:
https://github.com/user-attachments/assets/c91bf953-77d7-4c77-b5d7-b30d17a42ac5
Feel free to revert the change and open an issue instead to not spend too much time on it ;)

Feel free to revert the change and open an issue instead to not spend too much time on it ;)

I confirm, I think it's better so that we ensure to integrate this fix in v1.12 of Meilisearch 😊

Sorry for the oversight again. I tried to fix #565 to keep gaining familiarity with the code base.
If this is not working, I'll revert the changes so we can publish ASAP.


Use the Gateaux derivative rather than automatic differentiation

This is a major update that simplifies Phaseflow and makes everything except for 1D problems work with MPI.
It became apparent when testing MPI that our custom Newton solver was written as a serial algorithm, and I had no desire to write a parallel Newton solver. So the goal became to only use the nonlinear solver in fenics.
Until now, we only used fenics.derivative to automatically derive the Jacobian to solve the nonlinear problem. This has proven to be slow and not very robust.
Now I derived the proper Gateaux derivative and provide that directly to the nonlinear solver, so automatic differentiation isn't required. This is much faster and more robust. Also this makes it easy to map the equations in Phaseflow to the danaila2014newton paper, and also allows us to directly work with the differentiated nonlinear terms of interest, i.e. the Dirac delta functions from differentiating tanh.

Here's my derivation of the Gateaux derivative

It's still unclear why the builds for commits e36d10fd5b6090ed05460fd73e69d804a5d4c746  and 5514b99245ab60eb334055c7a08f44851ee3cf85  failed. I documented this in issue #64.


Implement RadxMars OTC De-Identification Support

User Story
As the CDC receiving RadxMars OTC data,
I want the message to be de-identified per the RadxMars rules,
so I can adhere to regulations
Description/Use Case
All messages going to the CDC receiver of RadxMars OTC data must be de-identified, per the RadxMars de-identification rules.
See the software requirements document section for more details:
II. Translation support
Risks/Impacts/Considerations
Dev Notes
Acceptance Criteria

[ ] RADxMARS OTC 2.5.1 messages can be de-identified on a per-receiver basis

blocked by prod requirements @brandonnava - FYI

@dylansmith-Nava


etcdserver: not print password in the warning message of expensive re…

…quest
Fix https://github.com/coreos/etcd/issues/9635
Could you try it, @YeruchamB ?

lgtm /cc @xiang90

@mitake I can still see password from user add command. Should we redact it for all AuthUser* requests?

Cherry-picked in https://github.com/coreos/etcd/pull/9661.


Add Scenarios Folder, Alternate Entrypoints Folder

Updated to create new Scenarios and Entrypoints folders to facilitate launching Scenarios. Includes sample entrypoints. Only for 0.16

@dtandersen any interest in this patch?

Made a couple of tweaks to this in https://github.com/dtandersen/docker_factorio_server/pull/139.
There is some duplication in the various startup scripts. Not sure if its worth fixing.

Cool!


Provide some instruction on how to update the azure.json file

The relationship between the fields in the azure.json file and the service principal fields are not immediately clear. For example aadClientId = appId and aadClientSecret = password.
https://github.com/kubernetes-incubator/external-dns/blob/master/docs/tutorials/azure.md#optional-create-service-principal has some more details. But without a lot of Googling it is hard to follow the instruction:

Then, update /etc/kubernetes/azure.json with the new credentials on all cluster nodes, and restart the nodes.

hello @mcasperson We are investigating this issue now. Thank you for reporting it

@DiegoRamirez-MSFT Appologies, it was https://docs.microsoft.com/en-us/azure/container-service/kubernetes/container-service-kubernetes-service-principal.
Specifically the https://docs.microsoft.com/en-us/azure/container-service/kubernetes/container-service-kubernetes-service-principal#credential-expiration section.

Thanks for the update @mcasperson will get this issue assigned to the most appropriate resource

@mcasperson thanks for reaching out. I am working offline to see what we can add. Will update you shortly.

@mcasperson I would suggest you look at the following doc instead:
https://docs.microsoft.com/en-us/azure/aks/kubernetes-service-principal
For kubernetes based cluster you should actually move to AKS instead of ACS. The AKS-based doc is much clearer as to what parts of the SP output is needed and overall running kubernetes in AKS will provide you with a much smoother and more reliable experience.
I will close this for now but if you have additional questions just let me know.
CC @iainfoulds @dlepow


feat: list of icons refactored (#463) (#527)

fix #463
@Lucassec1

These two aren't being showed:

Thanks for your contribution 😄

I appreciate the opportunity to be able to do it


Reconcile Archive URL's from DD for FHIR gen and 3++ exposure of FHIR items

Will need to be completed before Term Server go-live.
04/01/21 - Run-through complete, testing to start now

21/06 - in progress.

Decision made by NO that we won't add archive URL's into the R4 items - as there is no long term guarantee that the url's will remain resolvable in the long term.


[Bug] Random Word API is not working

Investigation required to find the reason of mentioned issue and fix it.

I think there is an issue in the API itself.
https://random-words-api.vercel.app/word says -
{"word" : "Due to Excessive Traffic, the API is temporarily unavailable.
Please clone the repo and run the code locally or deploy to your vercel
account. " , "definition " : " https://github.com/mcnaveen/random-words-
api " , " pronunciation " : "https : / /t. me/mcnaveen " }

This is an issue from server side. The API for random word is down.
The response returned is -
{"word":"Due to Excessive Traffic, the API is temporarily unavailable. Please clone the repo and run the code locally or deploy to your vercel account.","definition":"https://github.com/mcnaveen/random-words-api","pronunciation":"https://t.me/mcnaveen"}

@Ayush783  yes. I also checked this from the server.

Are there any alternate services?

@danger-ahead I tried to find some but they dont have proper JSON to fetch data.


Support for Mbed TLS?

I was wondering whether there is interest in supporting Mbed TLS in Wakaama. Has someone been working on an integration already?

I do not know Mbed TLS, but a quick glance make me think that is for tcp, is that correct? As far as I know there is only coap and udp support in wakaama andsupport for DTLS using tinyDTLS. Is there any specific features that you need in Mbed TLS that is missing in tinyDTLS?

@tuve it's supporting DTLS too: https://tls.mbed.org/kb/how-to/dtls-tutorial

I think the main problem with tinyDTLs is overall lack of maintenance and 1.3 and fullfledge PKI support.
Also the lack of TLS support in TinyDTLs can be a problem, for example if you use HTTPS for your FOTA you will need an extra lib for TCP TLS.

Maybe we should look into support that then. To my knowledge there isn't anybody currently looking into this.

I already used mbedtls with wakaama. I think the question should be if tinydtls and/or mbedtls should be part of the wakaama core implementation in the future. tinydtls is currently only used in the examples and is not part of the core wakaama library.

Good question. Even though I think you should never opt out of using dtls, I don't think it should be part of the core but rather referenced as a submodule as it is done now with tinydtls. But maybe this also should include the option of using mbed tls. Any thoughts on this?

From a security point of view this is not ideal. Everyone who doesn't use the examples implements their own security layer. So these changes tend not to be pushed back directly. In addition, the implementations are copied and this leads to errors that could be avoided.
If this is not taken into account, it is of course not a problem.

Yes, that is a concern. Does anyone have any figures on the memory footprint of tinydtls and mbed tls, both compiled code and runtime?

@LukasKarel You mentioned that you have used Wakaama with Mbed TLS. Is that code somewhere in a branch or so?

@tuve Regarding a comparison between Mbed TLS and TinyDTLS. I don't have that data but the answer will heavily depend on the details of the comparison. The feature set is different between the two stacks. The biggest contributor to code size in TLS/DTLS stacks, in general, is the crypto itself -- not the TLS protocol. I have data to support this observation in https://arxiv.org/pdf/2011.12035.pdf (see Figure 8 and Figure 9).

@LukasKarel You mentioned that you have used Wakaama with Mbed TLS. Is that code somewhere in a branch or so?

These repos are unfortunately not publicly available. But it should be pretty straight forward to implement a running prototype based on the current examples. Maybe I have some old files around and could create a working draft later this week.
@tuve I have never worked with tinydtls but mbedtls is highly configureable. Supported algorithms could be configured on compile time and therefore should reduce the footprint. Same is possible for buffer sizes to reduce heap usage for example.

@hannestschofenig correct me if I'm wrong but It seems to me that mbedTLS also brings Connection ID which is a great feature for LWM2M, correct ?

These repos are unfortunately not publicly available. But it should be pretty straight forward to implement a running prototype based on the current examples. Maybe I have some old files around and could create a working draft later this week.

About integrating mbedTLS, I think providing an example could be a good start.

I think the question should be if tinydtls and/or mbedtls should be part of the wakaama core implementation in the future.

From a security point of view this is not ideal. Everyone who doesn't use the examples implements their own security layer. So these changes tend not to be pushed back directly. In addition, the implementations are copied and this leads to errors that could be avoided.

Very good question but not so easy to decide. I guess it depends on :

the scope wanted for the project
how much it is easy to make it part of core library but also to make it possible to use other TLS/DTLS library (if we still want to do that).
how much it is compatible with https://github.com/eclipse/wakaama/issues/494

Personally, I have no answer but I feel this are good questions for Wakaama Committers and Community.

@sbernard31 Yes, Mbed TLS provides CID support for DTLS 1.2.

These repos are unfortunately not publicly available. But it should be pretty straight forward to implement a running prototype based on the current examples. Maybe I have some old files around and could create a working draft later this week.

About integrating mbedTLS, I think providing an example could be a good start.

I provided a sample but keep in mind, it is only a sample.
My opinion: Before thinking about integrating DTLS into wakaama (even multiple libraries), I would discuss the possibility to use wakaama with different CoAP libraries/or even a MQTT/HTTP library and try to seperate the layers.

I think separation and layering in general is a better approach. Especially if we want to support anything other than CoAP, as it is right now I think that would be quite hard.

Yes, Mbed TLS provides CID support for DTLS 1.2.

Be careful, if you use it!
You must use the right IANA code-point for the extension, depending on which version of the DTLS 1.2 CID draft you are want to use, or you must use, because it's the only implemented in the mbedtls version you are using.
mbedTLS - Update DTLS CID implementation to comply with the "final" draft

I will submit a PR to the Mbed TLS repo to make this update.

Regarding supporting other CoAP libraries and other tranports than CoAP: I am wondering whether this would be scope of a different PR since the use of Mbed TLS (instead of TinyDTLS), as explored with this PR, happens at a very different layer and should be fairly independent.

Regarding supporting other CoAP libraries and other tranports than CoAP: I am wondering whether this would be scope of a different PR since the use of Mbed TLS (instead of TinyDTLS), as explored with this PR, happens at a very different layer and should be fairly independent.

@hannestschofenig  did you really want to put this comment here  ? or you rather wanted to add it to https://github.com/eclipse/wakaama/pull/626 ?
(or I maybe missed your point)

I have updated the Mbed TLS CID implementation: https://github.com/ARMmbed/mbedtls/pull/5061


Not able to install packages

When we run cli/index.js ppman install python, it throws an error requested package not found
We then debugged the code and found that package list is fetched from this url - https://github.com/engineer-man/piston/releases/download/pkgs/index
This url returns 404 currently

Same here, please fix.

Should be fixed now, some reason the index didn't build when a new package was merged.

It works now, thanks a lot my man :) 👍

@Kogs Did you clear some cache? the issue still exists for me

thanks, redeploying worked


Jittery scorlling, when displaying a lot of tags

Description
When scrolling a folder, containing a lot of other folders with tags, the scrolling is jittery, pausing after every few folders, then continuing. It finishes the scroll animation about 1 second after stopping the scrolling on mouse.
Steps To Reproduce

Create multiple folders (87 in my case)
Give every folder a tag
Scroll down or up fast with a single motion

Requirements
The scrolling does not jitter, stop after every few folders when folders have tags.
Files Version
3.3.0..0
Windows Version
22H2
Log File
debug.log

Closed by accident, my bad.

I looked into this further and it turns out it's a known issue with the WinAppSdk framework we use in Files. We're going to close this issue but I'm linking the relevant issues here so you can add a comment to let Microsoft know this is affecting your workflow and usage of Files.
https://github.com/microsoft/microsoft-ui-xaml/issues/9584
https://github.com/microsoft/microsoft-ui-xaml/issues/9308


Fix shap value calculation and plot

@bagustris ?
when i try this, i get 4 errors on our tests, eg:
DEBUG: explore: running exp_emodb_explore from config tests/exp_emodb_explore_data.ini, nkululeko version 0.89.1
DEBUG: experiment: value for type not found, using default: audformat
DEBUG: dataset: emodb: loading from ./data/emodb/emodb
DEBUG: dataset: emodb: reusing previously stored file ./tests/results/exp_emodb_explore/./store/emodb.pkl
DEBUG: dataset: emodb: loaded with 100 samples: got targets: True, got speakers: True (10), got sexes: True
DEBUG: datafilter: emodb: limited samples to 100, reduced samples from 100 to 100
DEBUG: experiment: target: emotion
DEBUG: experiment: Labels (from database): ['neutral', 'disgust', 'sadness', 'fear', 'happiness', 'boredom', 'anger']
DEBUG: experiment: loaded databases emodb
DEBUG: experiment: reusing previously stored ./tests/results/exp_emodb_explore/./store/testdf.csv and ./tests/results/exp_emodb_explore/./store/traindf.csv
DEBUG: experiment: value for type is not found, using default: dummy
DEBUG: experiment: Categories test (nd.array): ['fear' 'neutral' 'anger' 'boredom' 'happiness' 'disgust']
DEBUG: experiment: Categories train (nd.array): ['happiness' 'boredom' 'anger' 'sadness' 'fear' 'disgust' 'neutral']
DEBUG: experiment: 10 speakers in test and 10 speakers in train
DEBUG: explore: train shape : (80, 8), test shape:(20, 8)
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/audeering.local/fburkhardt/research/nkululeko/nkululeko/explore.py", line 105, in <module>
    main(cwd)  # use this if you want to state the config file path on command line
  File "/home/audeering.local/fburkhardt/research/nkululeko/nkululeko/explore.py", line 95, in main
    expr.init_runmanager()
  File "/home/audeering.local/fburkhardt/research/nkululeko/nkululeko/experiment.py", line 508, in init_runmanager
    self.df_train, self.df_test, self.feats_train, self.feats_test
AttributeError: 'Experiment' object has no attribute 'feats_train'

I'll check...

With my recent push, there is no error on running test.
./run_test2.sh -spotlight

DEBUG: explore: running exp_polish_os from config tests/exp_polish_gmm.ini, nkululeko version 0.89.1
DEBUG: experiment: sampling selection: all
DEBUG: model: value for n_jobs is not found, using default: 8
DEBUG: feats_analyser: loading ./tests/results/exp_polish_os/./store//my_shap_values.pkl
DEBUG: feats_analyser: plotted SHAP feature importance to ./tests/results/exp_polish_os/./images/run_0/../train_dev_test_SHAP_gmm.png
DEBUG: experiment: value for scatter.target is not found, using default: ['class_label']
DONE
Total tests passed: 44
Total tests failed: 0
Total time taken: 235 seconds


fix: improve handling of at mentions

The extractMentionQuery function has been updated to handle cases where the user types the file without tabbing the selection. If a context selection is available and the current chat context query matches the input query, the first context selection should be automatically selected.
Test plan

Green bots = 🆗

at-mention a file without using tab or enter, just type out the whole file path, and end it with space
it should stlil include the file on submit

After

Before

@toolmantim not sure if this is a bug, or expected behavior. if it's expected, we can just close this PR 🙇♀


select produces different results than gen, even though the maximum likelihood answer should be the same (tokenization/token healing issue?)

The bug
I have a minimal reproducible example where I would expect select and gen to produce similar results, but they don't. My experimentation suggests maybe a tokenization or token healing issue, but I'm not sure. If the behaviour is expected, it would be useful to have some documentation to better understand why.
To Reproduce
from transformers import AutoModelForCausalLM, AutoTokenizer

import guidance

print("guidance version: ", guidance.__version__)

model_name = "unsloth/llama-3-70b-Instruct-bnb-4bit"
device = "cuda"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
lm = guidance.models.Transformers(model=model, tokenizer=tokenizer)

messages = [
    {"role": "system",
     "content": "You are a book information generator. Generate a JSON structure representing a book with the following properties: title, author, and publication date. Generate the author property first."},
    {"role": "user",
     "content": "The book is called 'The Great Gatsby', written by F. Scott Fitzgerald, and was published in 1925."},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

lm += prompt
lm += "{\n  "
# lm += guidance.select(["\"author\"", "\"title\""]) + guidance.gen(max_tokens=10)
lm += guidance.gen(max_tokens=10)
print(lm)

With select, the next output is "title" ("wrong" in a certain sense) while with unconstrained generation the output is "author" ("correct" in a certain sense).
 Generate a JSON structure representing a book with the following properties: title, author, and publication date. Generate the author property first.<|eot_id|><|start_header_id|>user<|end_header_id|>

The book is called 'The Great Gatsby', written by F. Scott Fitzgerald, and was published in 1925.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{
  "author": "F. Scott Fitzgerald",

(2) select output:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a book information generator. Generate a JSON structure representing a book with the following properties: title, author, and publication date. Generate the author property first.<|eot_id|><|start_header_id|>user<|end_header_id|>

The book is called 'The Great Gatsby', written by F. Scott Fitzgerald, and was published in 1925.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{
  "title": "The Great Gatsby",
   "author

System info (please complete the following information):

OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): RHEL 9
Guidance Version (guidance.__version__): 0.1.15

That's interesting. Is this happening with other models too?

If there's a model family you'd like me to try, feel free to let me know and I'll poke at it some more.

The select vs gen discrepancy happens with:

L3: unsloth/llama-3-70b-Instruct-bnb-4bit
L3: meta-llama/meta-llama-3-8B-Instruct (so it's not internal unsloth modifications causing issues, at least)

The whitespace issue happens with:

Phi: microsoft/phi-2

I've also tried use_fast=False on the tokenizer just in case and that doesn't seem to do anything.

And here are my experiments that suggest that this relates to the need to heal tokens around the "boundaries" between prompting and generation:
>>> tokenizer.encode(prompt + "{\n  ")
[128000, 128000, 128006, 9125, 128007, 271, 2675, 527, 264, 2363, 2038, 14143, 13, 20400, 264, 4823, 6070, 14393, 264, 2363, 449, 279, 2768, 6012, 25, 2316, 11, 3229, 11, 323, 17009, 2457, 13, 20400, 279, 3229, 3424, 1176, 13, 128009, 128006, 882, 128007, 271, 791, 2363, 374, 2663, 364, 791, 8681, 480, 36614, 518, 5439, 555, 435, 13, 10016, 62314, 11, 323, 574, 4756, 304, 220, 5926, 20, 13, 128009, 128006, 78191, 128007, 271, 517, 256]
>>> tokenizer.encode(prompt + "{\n  \"author\"")
[128000, 128000, 128006, 9125, 128007, 271, 2675, 527, 264, 2363, 2038, 14143, 13, 20400, 264, 4823, 6070, 14393, 264, 2363, 449, 279, 2768, 6012, 25, 2316, 11, 3229, 11, 323, 17009, 2457, 13, 20400, 279, 3229, 3424, 1176, 13, 128009, 128006, 882, 128007, 271, 791, 2363, 374, 2663, 364, 791, 8681, 480, 36614, 518, 5439, 555, 435, 13, 10016, 62314, 11, 323, 574, 4756, 304, 220, 5926, 20, 13, 128009, 128006, 78191, 128007, 271, 517, 220, 330, 3170, 1]
>>> tokenizer.encode(prompt + "{\n  \"title\"")
[128000, 128000, 128006, 9125, 128007, 271, 2675, 527, 264, 2363, 2038, 14143, 13, 20400, 264, 4823, 6070, 14393, 264, 2363, 449, 279, 2768, 6012, 25, 2316, 11, 3229, 11, 323, 17009, 2457, 13, 20400, 279, 3229, 3424, 1176, 13, 128009, 128006, 882, 128007, 271, 791, 2363, 374, 2663, 364, 791, 8681, 480, 36614, 518, 5439, 555, 435, 13, 10016, 62314, 11, 323, 574, 4756, 304, 220, 5926, 20, 13, 128009, 128006, 78191, 128007, 271, 517, 220, 330, 2150, 1]

LlamaCpp produces the same results as transformers.

from transformers import AutoModelForCausalLM, AutoTokenizer, PretrainedConfig

import guidance
from llama_cpp import Llama

print("guidance version: ", guidance.__version__)

model_name = "meta-llama/meta-llama-3-8B-Instruct"
# only for formatting the chat
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)
model_name = "bartowski/Meta-Llama-3-70B-Instruct-GGUF"

llm = Llama.from_pretrained(model_name,
                            filename="Meta-Llama-3-70B-Instruct-Q5_K_M.gguf",
                            n_gpu_layers=-1)
lm = guidance.models.LlamaCpp(llm)

if "llama" in model_name or "vicuna" in model_name or "aya" in model_name or "Llama" in model_name:
    messages = [
        {"role": "system",
         "content": "You are a book information generator. Generate a JSON structure representing a book with the following properties: title, author, and publication date. Generate the author property first."},
        {"role": "user",
         "content": "The book is called 'The Great Gatsby', written by F. Scott Fitzgerald, and was published in 1925."},
    ]
    prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
elif "phi" in model_name:
    prompt = "Instruct: Generate a JSON structure representing a book with the following properties: title, author, and publication date. The book is called 'The Great Gatsby', written by F. Scott Fitzgerald, and was published in 1925. Generate the author property first."
    prompt += "\nOutput: ```json\n"

lm += prompt
lm += "{\n  " + guidance.gen(max_tokens=100, name="generation")
# lm += "{\n  " + guidance.select(["\"title\"", "\"author\""], name="generation") + guidance.gen(max_tokens=100, name="generation2")
print(lm)

Thanks for the extra information. This is odd.

Yes, I think you're right! Here's an example where it still happens (this time, a discrepancy between two  gen calls with overlapping prompts), even without token healing being a factor (see the tokenization two replies up). When a leading quote is not provided, a property named author is generated, when it is, a property named _author is generated.

from transformers import AutoModelForCausalLM, AutoTokenizer

import guidance

from test_tokenization import print_tokens

print("guidance version: ", guidance.__version__)

model_name = "unsloth/llama-3-70b-Instruct-bnb-4bit"
device = "cuda"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
lm = guidance.models.Transformers(model=model, tokenizer=tokenizer)

messages = [
    {"role": "system",
     "content": "You are a book information generator. Generate a JSON structure representing a book with the following properties: `author`, `title`, and `publication_date`. Generate the author property first. All values should be strings."},
    {"role": "user",
     "content": "The book is called 'The Great Gatsby', written by F. Scott Fitzgerald, and was published in 1925."},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

total_prompt = prompt + "{\n  "

# lm += total_prompt
lm += total_prompt + '"'

lm += guidance.gen(max_tokens=10)

print(lm)

A quick note on severity/impact here -- I'm seeing this bug more often than not in my current project. Various workarounds (which amount to a kind of manual token healing as in the above example) are a bit finicky, so for my use case, this precludes me from using guidance. I'm transitioning to another todo item of mine for the next little while, but when I return I'll likely either (a) attempt bughunting in guidance, or (b) roll my own. In either case, I might learn something I can report back.
I know you guys are super busy, but if you or the rest of the guidance team have any tips for how I might approach (a) -- mainly where to start in grokking the internals of the library -- they'd be appreciated :) rolling my own would feel a bit silly. I'd ideally like to step through wherever guidance is doing tokenization with a debugger if you can point me in that direction.
Cheers and thanks again!

If you're really after JSON generation, then may I suggest our recently released JSON support:
https://guidance.readthedocs.io/en/latest/generated/guidance.json.html#guidance.json
Thank you for all the analysis you have done so far!

@wjn0 I want to take a closer look at what's going on behind the scenes RE: token healing, but in the meantime I second @riedgar-ms's suggestion to look at the built-in JSON support. Would you let us know if it behaves as you expect in this situation?
Also +1 to the thank you!

Thanks folks -- yes, absolutely, I love the JSON option when it's available to me. I haven't checked it for this minimal reproducible example (because it's just that for me -- a reduction of some stuff I was seeing in the real world), but I suspect it's OK based on earlier experimentation.
@hudson-ai @riedgar-ms In practice, I'm working with massive JSON schemas (several megabytes plain text w/ nested schemas) that seem to be far too large to compile to regexes (even w/ nested schemas removed + some preprocessing), so the hybrid template/structured generation approach has been ideal for me thus far, hence the example. Cheers again 👍🏻

Thanks folks -- yes, absolutely, I love the JSON option when it's available to me. I haven't checked it for this minimal reproducible example (because it's just that for me -- a reduction of some stuff I was seeing in the real world), but I suspect it's OK based on earlier experimentation.
@hudson-ai @riedgar-ms In practice, I'm working with massive JSON schemas (several megabytes plain text w/ nested schemas) that seem to be far too large to compile to regexes (even w/ nested schemas removed + some preprocessing), so the hybrid template/structured generation approach has been ideal for me thus far, hence the example. Cheers again 👍🏻

Thanks folks -- yes, absolutely, I love the JSON option when it's available to me. I haven't checked it for this minimal reproducible example (because it's just that for me -- a reduction of some stuff I was seeing in the real world), but I suspect it's OK based on earlier experimentation.
@hudson-ai @riedgar-ms In practice, I'm working with massive JSON schemas (several megabytes plain text w/ nested schemas) that seem to be far too large to compile to regexes (even w/ nested schemas removed + some preprocessing), so the hybrid template/structured generation approach has been ideal for me thus far, hence the example. Cheers again 👍🏻

We would be interested in hearing how the JSON support performs with large schema.

I have been doing a little digging, to which end I've created the following test case:
from guidance import models, select, gen, system, assistant, user

def prepare_model(lm: models.Model):
    with system():
        lm += "You are a book information generator. Respond with \"author\" or \"title\" followed by the value."
    with user():
        lm += "The book is called 'The Great Gatsby', written by F. Scott Fitzgerald"
    return lm

def test_with_gen(selected_model: models.Model):
    lm = prepare_model(selected_model)
    with assistant():
        lm += gen(max_tokens=100)
    print(lm)
    assert str(lm) == "Hello"

def test_with_select(selected_model: models.Model):
    lm = prepare_model(selected_model)
    with assistant():
        lm += select(["\"author\"", "\"title\""]) + gen(max_tokens=100)
    print(lm)
    assert str(lm) == "Hello"

The two tests obviously fail, but when running in the debugger I can break in _transformers.py::get_logits and take a look at the tokens passed in (for speed, I'm using GPT2)
They are identical, except that the select() variant has an extra " token appended. This is as it should be - both options to select() start with a double quote, so that can be inserted automatically. But that means that the actual prompts sent into the model are different.

Have just persuaded Phi3 to work with this (see #885 ), and the final LLM state is:
gen()
<|user|>Convert the following information into JSON with keys 'author' and 'title'. Put the title first. The book is 'The Great Gatsby' by Fitzgerald.<|end|><|assistant|>{
  "title": "The Great Gatsby",
  "author": "Fitzgerald"
}<|end|>

and
select()
<|user|>Convert the following information into JSON with keys 'author' and 'title'. Put the title first. The book is 'The Great Gatsby' by Fitzgerald.<|end|><|assistant|>{
"title": "The Great Gatsby",
"author": "Fitzgerald"
}<|end|>

so, same JSON document, but different spacing.
Digging into the first call to get_logits(), I get:
gen():
Last tokens: [29915, 491, 22963, 914, 2741, 29889, 32007, 32001, 29912, 13] forced_bytes=b''

and
select()
Last tokens: [29915, 491, 22963, 914, 2741, 29889, 32007, 32001, 29912, 13] forced_bytes=b'"'

so, while the tokens being sent are the same, again select() is constraining the output so that it has to start with a double quote (as we should expect).

Ok, neat! I had assumed that guidance internally was doing something similar to outlines w.r.t. regex-based JSON generation, but after looking at the code more closely it looks like the guidance implementation is "lazier" and therefore might actually work for my usecase! Super exciting, will give it a shot.
I'll take a closer look at the tokenization issue with fresh eyes as well. Thanks for the pointers on that part of the library, I'll drill down there when I next get the chance.

If you have large JSON schema, we'd be really interested to know how Guidance performs. We only have functionality tests; we've not really tried to stress our implementation.
Do let us know if there are gaps in the implementation too.

@riedgar-ms @hudson-ai So, I gave it a shot. wjn0/guidance@improve-json-schema-support contains a few hackish changes that I required for my schema. These are not legitimate fixes (i.e. you wouldn't want these as PRs:) so I've created feature request issues for each:

#887
#888

with some discussion items. Sadly, although this prevents errors, it still (understandably) hangs on a huge schema (hopefully not due to bugs I've introduced with my hackish fixes:'). Therefore, I've also created:

#889

with some thoughts on strategy, along with an example. There's still some gaps compared to my templating approach I've been working on mentioned above ^ that I've created as separate issues:

#890
#891

My intention here is not to drown the repo in feature requests or issues, but it's certainly moved beyond #876 alone and I think this is a reasonable breakdown. Obviously, please feel fee to close/consolidate as appropriate. This would cover my use case (and the root case of the ticket here). Cheers!

@wjn0 please don't feel self-conscious about inundating us with issues -- I think all the issues you've opened represent really valid requests and start good discussions. We appreciate your engagement :)


added initial docs

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/513/
Test PASSed.


Feature add target blank

Issue:
https://github.com/arvind0598/better-first-issues/issues/20

Thanks for your help :)


IGAPP-969: Default social media preview

This pull request belongs to an issue on our bugtracker.
You can find it there by looking for an issue with the key which is mentioned in the title of this pull request.
It starts with the keyword IGAPP.

can i test this somehow locally?

Testable by sharing https://test.integreat.app on facebook


PageRenderType is InvalidFieldTypeInForm on NewForm when a TaxonomyField on the list is not shown on the Form

When setting ShowInNewForm to false for a Taxonomy field on a list, the PageRenderType of the NewForm returns InvalidFieldTypeInForm.
This causes the built-in auto-detect to switch the rendering of the NewForm back to classic.
If the Taxonomy field is shown on the NewForm, the PageRenderType for the Form returns Modern and the rendering doesn't switch back to classic.
This doesn't seem to be documented anywhere so is a bug or is it not supported to render a Modern NewForm when there is a Taxonomy field on the list which is not shown in the Form?
(Setting ReadOnly to true for a Taxonomy field triggers the same behaviour)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4245fa14-c317-5a0f-7aba-6569d5cb268d
Version Independent ID: 2a351a4d-3600-b80a-6ae3-8b1a494d69c7
Content: Customizing 'modern' lists and libraries
Content Source: docs/solution-guidance/modern-experience-customizations-customize-lists-and-libraries.md
Product: sharepoint
GitHub Login: @VesaJuvonen
Microsoft Alias: vesaj

@VesaJuvonen I see this issue is now closed and marked as an outdated issue.
Does this mean it's fixed or is it the intended behaviour for SharePoint to render classic NewForm when there is a Taxonomy field on the list which is not shown in the Form?


muting: Correct agrammatical error message.

The error message displayed when unmuting a topic that wasn't previously muted wasn't properly formulated.
Followup for #10345.

Testing Plan: ./tools/test-backend zerver.tests.test_muting passed locally.

Merged, thanks @YagoGG!


Define desired kubernetes object names

The builder needs to use suitably unique names for the kubernetes "stuff" required for running chaincode
The run binary has access to the unique ID associated with the chaincode package, which is in the form of label:hash, the address of the peer, which is in the form of host:port, and the local mspid of the peer.
The builder can also use environment variables which are configured in core.yaml, for example CORE_PEER_ID can be propagated to the builder, and currently is required. Using the peer address would reduce the configuration required and may be preferable.
Important: whatever combination is used needs to be sanitised and truncated to give a valid kubernetes object name, and the naming rules differ between objects. For example the peer ID can be relatively long, e.g. http://peer0.org1.example.com/ in the nano test network, and the same for the chaincode label, plus the chaincode ID includes a full hash. All these will need to be truncated and have any invalid characters removed or replaced.

This should fix the following problems (which need tests!)

secret name...

unable to create kubernetes secret for chaincode ID CHAINCODE_ID: Secret "cc-mspid-core_peer_id_abcdefghijklmnopqrstuvwxyz_0123456789chai" is invalid: metadata.name: Invalid value: "cc-mspid-core_peer_id_abcdefghijklmnopqrstuvwxyz_0123456789chai": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')

util.mangleName...

panic: runtime error: slice bounds out of range [:63] with length 33


Remove engine dependency

I've inserted flutter_embedding_debug dependency for android studio intellisense, but it looks like it can break build with new engine versions. So I replaced it with no version specified one, and specified version only when opening this project directly with android studio.
Intellisense has no problem by opening example/android directory. So remove redundant dependency.

Thanks for contribution!


Fix calling reference on polymorphic relationships

Purpose or Intent
Fixes #10450
For the record:  I am not really proud or satisfied with the code added here in it's current state, but it fixes the problem it seems.
Overview
Recently, the `lib/extensions/ar_merge_conditions.rb file was removed from the repo as it was deemed no longer to be needed, and we had converted everything over to using AREL.
The main method that was removed was the following:
def self.apply_legacy_finder_options(options)
  unknown_keys = options.keys - LEGACY_FINDER_METHODS.map(&:first)
  raise "Unsupported options #{unknown_keys}" unless unknown_keys.empty?

  # Determine whether any of the included associations are polymorphic
  has_polymorphic = included_associations(options[:include]).any? { |name| self.reflection_is_polymorphic?(name) }

  LEGACY_FINDER_METHODS.inject(all) do |scope, (key, method)|
    # Don't call references method on scope if polymorphic associations are
    # included to avoid ActiveRecord::EagerLoadPolymorphicError
    next(scope) if method == :references && has_polymorphic
    #
    options[key] ? scope.send(method, options[key]) : scope
  end
end

Which basically mapped the old Rails 2.3 methods (LEGACY_FINDER_METHODS, not shown) to the new(er) Rails 3+ AREL syntax.  The important part of note is the lines with has_polymorphic, which prevents applying the values from the :include key (which is mapped to both the .includes and .reference methods) to the .reference method.
When the refactoring was done in 062263c (part of the https://github.com/ManageIQ/manageiq/pull/10175 pull request), the check for the reference methods was not brought along, so when an invalid :include_for_find key was passed in with a polymorphic relationship, the code would break with a not so obvious error message:
Error caught: [ActiveRecord::ConfigurationError] nil

Most of the code in ar_merge_conditions.rb is actually for properly determining when a polymorphic relationship exists, so in reality, this code maybe should have stayed in some way or another.  The changes here are an attempt to reduce the amount of code added, but it might be to limited in what keys are checked.
Regardless, it does prevent the error.
Links

https://github.com/ManageIQ/manageiq/issues/10450
https://github.com/ManageIQ/manageiq/pull/10175
https://github.com/ManageIQ/manageiq/blob/6377ca3/lib/extensions/ar_merge_conditions.rb

Steps for Testing/QA
Well, you can easily reproduce this error without Rbac by doing the following:
MetricRollup.includes(:resource => {}).references(:resource => {})

With Rbac, see the tests for an example, but it is very similar.
This can be recreated by viewing Cap & U data for a Host.  See #10450 for tips on recreating locally.

:+1: LGTM. @gtanzillo this occurred when we removed apply_legacy_options, specifically with references().
The reports have include that specify columns. This is typically ok, but a problem if the include is a polymorphic column.


⚠ Quirk Club Argentina (API endpoint) has degraded performance

In 8f8c250, Quirk Club Argentina (API endpoint) (https://us-central1-quirkclub-dev.cloudfunctions.net/api/check/api) experienced degraded performance:

HTTP code: 200
Response time: 8340 ms

Resolved: Quirk Club Argentina (API endpoint) performance has improved in 17edc76.


Update move-database-between-pools.sh

updating to follow new style guide

@jan-eng,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


Open Bible when clicking

Hi,
When I click the bible text to read, it does not open that bible part for that day to read.
Is possible to open bible when clicking?

Hi Pedro,
Thank you for reporting. That's unusual...
Is it just this one text? Is it just for Joshua? Is it just on this schedule?
It is quite a large portion, perhaps it takes a while to "compile" into the side pane... What kind of device are you using?
Any and all information would be greatly appreciated ;-)

In a windows pc, I cannot open any bible link. It does not load any info.
In a surface go tablet (windows), takes some time but it loads.
In iphone it works correctly.

I just tried the Portuguese version on my Windows 10 setup and it worked.
What Bible translations do you have loaded/installled? Can you try removing the BRS publication, and adding it again, please?

Resolved.


Typescript errors when trying to deploy function

Environment
Operating System version: Windows 11
Browser version: Google chrome 109.0.5414.75
Firebase library: Firebase and version: 10.2.0
GeoFirestore version: 5.2.0

Getting error in firebase deploy cmd
when i trying to deploy the function getting error i tried by deleting node_modules folder and then npm install but it's not resolved.
`
i  deploying functions
Running command: npm --prefix "$RESOURCE_DIR" run build

build
tsc

node_modules/@google-cloud/firestore/types/firestore.d.ts:23:1 - error TS6200: Definitions of the following identifiers conflict with those in another file: DocumentData, UpdateData, Firestore, GeoPoint, Transaction, BulkWriter, BulkWriterError, WriteBatch, SetOptions, WriteResult, DocumentReference, DocumentSnapshot, QueryDocumentSnapshot, OrderByDirection, WhereFilterOp, Query, QuerySnapshot, DocumentChangeType, CollectionReference, CollectionGroup, QueryPartition, FieldValue, FieldPath, Timestamp, BundleBuilder, v1beta1, v1, OK, CANCELLED, UNKNOWN, INVALID_ARGUMENT, DEADLINE_EXCEEDED, NOT_FOUND, ALREADY_EXISTS, PERMISSION_DENIED, RESOURCE_EXHAUSTED, FAILED_PRECONDITION, ABORTED, OUT_OF_RANGE, UNIMPLEMENTED, INTERNAL, UNAVAILABLE, DATA_LOSS, UNAUTHENTICATED, FirebaseFirestore
23 declare namespace FirebaseFirestore {

node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:23:1
 23 declare namespace FirebaseFirestore {
    ~~~~~~~
 Conflicts are in this file.

node_modules/@google-cloud/firestore/types/firestore.d.ts:174:5 - error TS2374: Duplicate index signature for type 'string'.

174     [key: string]: any; // Accept other properties, such as GRPC settings.
     ~~~~~~~~~~~~~~~~~~~

node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:23:1 - error TS6200: Definitions of the following identifiers conflict with those in another file: DocumentData, UpdateData, Firestore, GeoPoint, Transaction, BulkWriter, BulkWriterError, WriteBatch, SetOptions, WriteResult, DocumentReference, DocumentSnapshot, QueryDocumentSnapshot, OrderByDirection, WhereFilterOp, Query, QuerySnapshot, DocumentChangeType, CollectionReference, CollectionGroup, QueryPartition, FieldValue, FieldPath, Timestamp, BundleBuilder, v1beta1, v1, OK, CANCELLED, UNKNOWN, INVALID_ARGUMENT, DEADLINE_EXCEEDED, NOT_FOUND, ALREADY_EXISTS, PERMISSION_DENIED, RESOURCE_EXHAUSTED, FAILED_PRECONDITION, ABORTED, OUT_OF_RANGE, UNIMPLEMENTED, INTERNAL, UNAVAILABLE, DATA_LOSS, UNAUTHENTICATED, FirebaseFirestore

23 declare namespace FirebaseFirestore {

node_modules/@google-cloud/firestore/types/firestore.d.ts:23:1
23 declare namespace FirebaseFirestore {
~~~~~~~
Conflicts are in this file.
node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:71:25 - error TS2315: Type 'UpdateData' is not generic.
71     ? {[K in keyof T]?: UpdateData<T[K]> | FieldValue} & NestedUpdateFields
~~~~~~~~~~~~~~~~
node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:104:28 - error TS2315: Type 'UpdateData' is not generic.
104         AddPrefixToKeys<K, UpdateData>
~~~~~~~~~~~~~
node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:282:5 - error TS2374: Duplicate index signature for type 'string'.
282     [key: string]: any; // Accept other properties, such as GRPC settings.
~~~~~~~~~~~~~~~~~~~
node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:655:13 - error TS2315: Type 'UpdateData' is not generic.
655       data: UpdateData,
~~~~~~~~~~~~~
node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:806:13 - error TS2315: Type 'UpdateData' is not generic.
806       data: UpdateData,
~~~~~~~~~~~~~
node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:1029:13 - error TS2315: Type 'UpdateData' is not generic.
1029       data: UpdateData,
~~~~~~~~~~~~~
node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:1254:13 - error TS2315: Type 'UpdateData' is not generic.
1254       data: UpdateData,
~~~~~~~~~~~~~
Found 10 errors in 2 files.
Errors  Files
2  node_modules/@google-cloud/firestore/types/firestore.d.ts:23
8  node_modules/geofirestore-core/node_modules/@google-cloud/firestore/types/firestore.d.ts:23
Error: functions predeploy error: Command terminated with non-zero exit code 2`
package.json
"engines": {
"node": "16"
},
"main": "lib/index.js",
"dependencies": {
"express": "^4.18.2",
"firebase-admin": "^10.2.0",
"firebase-functions": "^3.21.0",
"geofirestore": "^5.2.0",
"moment-timezone": "^0.5.40",
"ngeohash": "^0.6.3",
"uuid": "^9.0.0"
},
"devDependencies": {
"typescript": "^4.6.4"
},
My Code
import { Request, Response, NextFunction } from "express";
import * as admin from "firebase-admin";
import * as geofirestore from "geofirestore";
//let GeoFire = require('geofire');
var geohash = require("ngeohash");
export async function filterBusinessByDistance(req: Request, res: Response) {
try {
const uid = res.locals.uid;
const latitude = 28.431399;
const longitude = 77.50627;
const firestore = admin.firestore();
const Geofirestore = geofirestore.initializeApp(firestore);
const geocollection = Geofirestore.collection("business");
const queryCollection = geocollection.near({
  center: new admin.firestore.GeoPoint(latitude, longitude),
  radius: 20,
  limit: 35,
});
let businessNearbyRef = await queryCollection.get();

res.status(200).json({
  status: "1",
  message: "success",
  payload: businessNearbyRef,
});

} catch (error) {
console.log(error);
res.status(400).json({
status: "0",
message: error,
payload: null,
});
}
}

I'm also getting this problem.
This is serious, the library is not being maintained and updated. No work around as far as I know.

I'm also getting this problem.
This is serious, the library is not being maintained and updated. No workaround as far as I know.

switch to geofire-common . supported by google. since geofirestore stores geohash, just point geofire-common to it. so there is no need to change the firestore db.

import * as geofire from 'geofire-common';

/**
   * Given a firebase Query, add a check to get the range bound entries.
   * Reuse lat, lng attributes defined by now deprecated geofirestore
   * @param userCoords geofire.GeoPoint
   * @param _radius in metres
   * @return array of promises of of QueryDocumentSnapshots that match the distance specified
   */
  getGeoQuery(
    geoCollection: FirebaseFirestore.CollectionReference,
    userCoords: geofire.Geopoint,
    _radius: number): Observable<FirebaseFirestore.QueryDocumentSnapshot[]> {
    const bounds = geofire.geohashQueryBounds(userCoords, _radius);
    const promises = <Promise<FirebaseFirestore.QuerySnapshot>[]>[];

    for (const b of bounds) {
      const q = geoCollection.orderBy('g.geohash').startAt(b[0]).endAt(b[1]);

      promises.push(q.get());
    }
    return from(Promise.all(promises).then((snapshots) => {
      const matchingDocs = [];

      for (const snap of snapshots) {
        for (const doc of snap.docs) {
          const coords = <GeoPoint>doc.get('geoCoords');

          const distInKm = geofire.distanceBetween([ coords.latitude, coords.longitude ], userCoords);
          const distInM = distInKm * 1000;

          if (distInM < _radius) {
            matchingDocs.push(doc);
          }
        }
      }
      return matchingDocs;
    }));
  }


Apple device specific link images to data urls

Implemented simply by adding targets to favicon list as it is already used also for other images that are not favicons.

Sorry, no idea why I missed this, thanks!


New EF MVC tutorial series

This is an update of existing series for EF 6 MVC 5 at
http://www.asp.net/mvc/overview/getting-started/getting-started-with-ef-using-mvc/creating-an-entity-framework-data-model-for-an-asp-net-mvc-application
/cc @Rick-Anderson @spboyer

@tdykstra can you fix the build issues?

Reopened as #1879


.sub() behaves differently on Node and Python for Unsigned Int

Hi,
I found maybe? a bug?
At least for Mat, substract Mat behaves differently under Python and Node (didn't test c++).
Since Python is using numpys, substracting overflow results in it wrapped, while on Node it ends as zero.
This creates very subtle and hard to track issues if you try to port Python code to Node code.
Tbh I'm not sure if it's a bug, but if it's not, it might be worth mentioning in examples/docs
Python:
import cv2
import numpy as np
m = np.zeros((1, 1), dtype = "uint8"), n = np.zeros((1, 1), dtype = "uint8")
m[0][0] = 2; n[0][0] = 4
print((m - n)[0][0])

results in 254
Node:
const cv = require('@u4/opencv4nodejs');
let m = new cv.Mat(1, 1, cv.CV_8U); let n = new cv.Mat(1, 1, cv.CV_8U);
m.set(0, 0, 2); n.set(0, 0, 4)
console.log(m.sub(n).at(0, 0))

results in 0
To give a concrete example of when it quietly fails, Python examples for edge detection will include double gausianBlur with different strengths and the subtraction of those will be vastly different.
OpenCV version 4.5.5
With OpenCV-contrib? (extra modules): yes
OS: MacOSX

Hi,
I made some minor changes to your message; With some lint + color, it is more pleasant to read.

Hi, thank you for prompt reply. Unfortunately, turns out it's a feature:
https://docs.opencv.org/4.x/d0/d86/tutorial_py_image_arithmetics.html
There is a difference between OpenCV addition and Numpy addition. OpenCV addition is a saturated operation while Numpy addition is a modulo operation.
What that means though, is this example:
https://github.com/UrielCh/opencv4nodejs/blob/be727d680b13b4f143b5e14ff449dc5792709826/examples/src/getStructureSimilarity.ts#L27
Will behave differently than the Python original version.
I'll update title to reflect that and we can close. I couldn't find an easy way how to replicate - behaviour with OpenCV, as there's no builtin modulo operation, and trying to replicate it with division doesn't work because for modulo you need floor while mat.div() uses round :(

for references:
https://github.com/UrielCh/opencv4nodejs/blob/master/cc/core/coreUtils.h


Pretzel should have an installer

An installer would eliminate the need for users to add pretzel to their path (https://github.com/Code52/pretzel/wiki/Installing-Pretzel).

I think it can be closed now: https://chocolatey.org/packages/pretzel.

Thanks, I haven't reviewed all the old issues yet :smiley:


Support panning with the mouse wheel.

The movement scheme is based on common browsers, like Firefox and Chromium, assumed to be a good approximation of what users find intuitive.
I realize that would be a change WRT the current behavior, and honestly I'm not too attached to it. I can switch to any other combination for the modifier keys if that's preferred.

@Plagman Do you have any thoughts on this one Pierre-Loup? Thanks!

Not convinced it makes sense to change as the default, given you're often constantly scrolling in X/Y directions and need quick access to zoom in/out as well. I think the current model matches well with usage, and would be hard to change from under people who are used to it. Might make sense to have as an option, or with a modifier key?

Not convinced it makes sense to change as the default, given you're often constantly scrolling in X/Y directions and need quick access to zoom in/out as well. I think the current model matches well with usage, and would be hard to change from under people who are used to it. Might make sense to have as an option, or with a modifier key?

I can probably adapt to any other modifier combination. What would your favorite one be? Basically you have to choose a modifier (including no modifier) for each of zoom, pan horizontally and pan vertically.


Best practice for creating a local dask cluster?

I'm not sure that creating a local cluster with with n_workers=os.cpu_count() is best practice. First, os.cpu_count() can return None, and it's not clear what n_workers=None would do. Secondly, os.cpu_count() returns the number of CPUs on the system, which may not be the same as the number of cores available to the process. Thus os.sched_getaffinity(0) might be a better choice. However, though the Dask documentation on LocalCluster is incomplete, the best choice may simply be to let LocalCluster figure out the value itself. See discussion here.
cluster = LocalCluster(n_workers=os.cpu_count())
client = Client(cluster)
client

https://github.com/ProjectPythia/interactive-sentinel-2-cookbook/blob/268f9e9a218819570eff8531c3b54b22b3f44a4b/notebooks/data-intake-ms-planetary-computer.ipynb#L94

I hadn't thought about this behavior of os.cpu_count() earlier, thanks for your review. I have opened a PR (#6) addressing this issue.

Fixed by #6


test(booleanAttrsSpec): add unit test for IE11 URL parsing failure

IE11/10/Edge fail when setting a href to a URL containing a % that isn't a valid escape sequence
Reference: https://github.com/angular/angular.js/issues/13388

If the point of this is to highlight a bug in IE then we should make that clear in the code as well as the commit and include a link to the issue. Otherwise when this starts to fail we will wonder why it was ever there.

Otherwise LGTM

b2b896f
1f8431b


Docs Request: Click to reveal code snippets

What is the issue you are encountering with the docs?
Suggesting a modification to the lesson structure. Instead of immediately showing the expected result after instructing people to code something on their own, I propose making some of the code blocks "click to reveal." This means learners would have the option to see the solution only when they choose to, creating a more engaging and interactive learning experience.
Describe the solution you'd like to see.
There are a lot of different ways to do this but a simple solution would be a collapsible section written in markdown.
https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-collapsed-sections#creating-a-collapsed-section

@Valcoholics nice idea. I'm also happy to pick this one up if you're not working on it already.


Add jwst_gtvt dependency to setup.py

According to Jeff Chamblee, It would be helpful to the ExoMAST builds if jwst_gtvt dependency could be listed in setup.py.
Currently, jwst_gtvt does not exist on pypi, however it should be doable by installing directly from GitHub.

so just moving git+https://github.com/spacetelescope/jwst_gtvt.git to setup.py? @bourque

@jaymedina Yes, I believe so


chore: update Electron 28

📝 Description
Maintenance update of Electron on LLD.
❓ Context

JIRA or GitHub link:  https://ledgerhq.atlassian.net/browse/LIVE-10360

✅ Checklist
Pull Requests must pass the CI and be code reviewed. Set as Draft if the PR is not ready.

[ ] npx changeset was attached.
[x] Covered by automatic tests.
[x] Impact of the changes:

there should not be any impact https://www.electronjs.org/fr/blog/electron-28-0

🧐 Checklist for the PR Reviewers

[ ] The code aligns with the requirements described in the linked JIRA or GitHub issue.
[ ] The PR description clearly documents the changes made and explains any technical trade-offs or design decisions.
[ ] There are no undocumented trade-offs, technical debt, or maintainability issues.
[ ] The PR has been tested thoroughly, and any potential edge cases have been considered and handled.
[ ] Any new dependencies have been justified and documented.
[ ] Performance considerations have been taken into account. (changes have been profiled or benchmarked if necessary)

@SocketSecurity ignore npm/<EMAIL_ADDRESS>

/generate-screenshots


Take new frames into account when tracking actions ordering

We defined actions ordering through regular component connections, now that frames don't create them, actions ordering was not taking frames into account.
Also fixes pasting components into a frame.

bors merge

bors merge


Add support for Swift 5.5 and Xcode 13

The dynamic type is no longer needed and actually breaks running tests for Danger plugins with Xcode 13.
The error that's fixed with this PR:

Swift package target 'Danger' is linked as a static library by 'WeTransferPRLinterTests' and 'Danger', but cannot be built dynamically because there is a package product with the same name.

Fixes #475

I've reproduced the issue here https://github.com/danger/swift/runs/4055493568?check_suite_focus=true and added a CI suite that can reproduce the issue

@f-meloni is there a middle ground you think that fixes the Xcode 13 bug ánd doesn't break old systems?
As a last resort we could keep using our own fork, but that's obviously not ideal!

@AvdLee I would love to find something like a parameter that I can pass to swift build to force it to build Danger as dynamic without having to specify it on SPM.
If that doesn't work then the only other solution I can think of is changing the Package.swift from the install script before installing

If that doesn't work then the only other solution I can think of is changing the Package.swift from the install script before installing

If it's only failing for that script, it would make total sense. It would be weird to me to not support SPM which is heavily used in Danger environments, so we need to get to a working solution. This sounds like the way to go to me!


cd.splunkdev.com dependencies

welp
https://github.com/splunk/kafka-mq-go/blob/105be9903bf04185b3dd0d6db40b6b346cc67c9f/queue/consumer.go#L8-L9
;)

Oh I see /go.mod declares this as living there too, so it can resolve them as relative... Might be good to change these to github.com/splunk/kafka-mq-go/... then ;)


Rescue NonExistingDocumentError in base server

Motivation
In #2313, I forgot that we need to rescue the new NonExistingDocumentError in base server, otherwise the TypeScript client tests will fail when VS Code opens some fake files automatically.
Implementation
Added the missing rescue.

Should there be a test case for this though? Otherwise it'll be hard to tell if/when we'll need this in the future.


Detectron v2 is not installed

When i run this code following: python remove_anything.py  --input_img ./example/remove-anything/dog.jpg --point_coords 200 450 --point_labels 1 --dilate_kernel_size 15 --output_dir ./results  --sam_model_type "vit_h" --sam_ckpt ./pretrained_models/sam_vit_h_4b8939.pth --lama_config ./lama/configs/prediction/default.yaml --lama_ckpt ./pretrained_models/big-lama, there have:

i keep following as the Installation and usage instruation, but i can't know what happend to get the "Detectron v2 is not installed" error?

And what is the "Detectron v2"? Is it Mmdetection v2?

When i run this code following: python remove_anything.py --input_img ./example/remove-anything/dog.jpg --point_coords 200 450 --point_labels 1 --dilate_kernel_size 15 --output_dir ./results --sam_model_type "vit_h" --sam_ckpt ./pretrained_models/sam_vit_h_4b8939.pth --lama_config ./lama/configs/prediction/default.yaml --lama_ckpt ./pretrained_models/big-lama, there have:
i keep following as the Installation and usage instruation, but i can't know what happend to get the "Detectron v2 is not installed" error?

Just ignore it. Refer to this issue.

Yeah!
I just ignore it, and then I can generate removed results.
In addition, i have a question:
Can't I automatically select the picture and mark the coordinate points through the code?
I can only manually enter the coordinates to eliminate objects, because I tested it on my own data, and the effect is good or bad, and the generalization ability of this "Segment Anything Model (SAM)" is also limited. (Hahaha...)

Yeah! I just ignore it, and then I can generate removed results. In addition, i have a question: Can't I automatically select the picture and mark the coordinate points through the code? I can only manually enter the coordinates to eliminate objects, because I tested it on my own data, and the effect is good or bad, and the generalization ability of this "Segment Anything Model (SAM)" is also limited. (Hahaha...)

Hi, that's exactly what we are working on. The demo website is coming soon. You can achieve it when it's released. Before that, you may need use get_ponit_coor.py to get the coordinates.
SAM performs better when you give more positive/negative points, and we're considering the multi-click feature.

I close this issue as completed.

but i have no results，

但我没有结果，

I do  ，sry

A link appeared, but it did not automatically redirect. After clicking, I found that the link could not be opened. How can I solve this problem?


Ashwin95525 patch 2

Added the git commands as code snippets.

@Ashwin95525 , kindly add steps to create a virtual environment and to activate it.

@GOPAL-YADAV-D, Sure I can do that. So should I just contribute under the Contribute.md file and just create a new branch?

@Ashwin95525,yes kindly make changes and commit to the same branch from which PR was created

@GOPAL-YADAV-D , Thank you for suggesting the idea-steps to create a virtual environment and to activate it. I wrote about it in the "create contributing.md" file and made a new pull request. I hope that will work!!
Thanks

Hi @Ashwin95525 , we will be merging this PR !
Your contributions are appreciated!
Great work

@varun10sudhir Thank you so much  for merging the PR and for your  time I really appreciate it.


pull request for e2e test at 11/23/2016 01:56:09

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Create e2e tests for the Aztec CLI

We have a few options on how to call the CLI from an e2e test suite:

Go through the shell. Use child_process.exec to call the CLI directly through the shell in the context of a jest test suite and parse the output. It's as e2endy as you can get, but cumbersome (and brittle) due to the parsing.
Get an instance of the configured program from commander and manually call program.parse with whatever arguments we want. We'd be running the CLI on the same process as the tests (which is probably faster? or at least removes the inconveniences of having to locate the CLI binary), but we'd still have the same issues with parsing output.
Split out the CLI logic, so yarn-project/aztec-cli/src/index.ts only declares the commands, and we keep separate functions with the actual logic that map 1:1 to each command. These functions can return stuff (instead of or in addition to) logging it, and we test these functions.

After discussing with @PhilWindle we're leaning towards option 2.

Possible duplicate of #1450?

Certainly!


Syntax errors when using file association "*.es": "javascriptreact"

VSCode Version: 1.8.1 (clean, without extensions)
OS Version: Windows 10

Steps to Reproduce:

Create file association ("*.es": "javascriptreact") in user settings
Paste some React code

I have no errors with same code in ".js" file.

Thanks for reporting! Looks like this is a duplicate of https://github.com/Microsoft/vscode/issues/15460, so I'm going to go ahead and close this issue. Let me know if you think otherwise, and we'll be happy to reopen. Happy coding!


hello-oboe: Allow explicit selection of audio API (OpenSLES or AAudio)

Thanks Phil! I'll address these on the master branch since this was merged already.

Addressed here: https://github.com/googlesamples/android-audio-high-performance/commit/7f9731f815061b257ee5899baec38f1a85567baa


Merge cri branch into master

@squeed @jonboulle @lucab Having https://github.com/appc/spec/pull/663 landed, do we feel confident enough to merge the appc/spec cri branch into appc/spec master?
That would allow us to have only one cri working branch in rkt.

Done in #667, this can be closed


fixed grammar issue on about page

Description

Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue

Change Status

[x] Complete, tested, ready to review and merge

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] My changes generate no new warnings
[x] New and existing unit tests pass locally with my changes
[x] There are no merge conflicts
[ ]

Johnny, thank you for tackling this very necessary sweep of the about section. Don't let the number of my comments dissuade you, you're doing good work, we just have a few things to address before merging. If you have any questions and I don't appear to be responsive her, please feel free to DM me on slack for more clarification. (I will try to check here first.)


Lens Blur format used in Depthy expects a reversed image.

I'm noticing that in Depthy it expects darker stuff to be closer to the screen. However what is extracted from the files that the iPhone provides are reversed. Darker stuff is further away. That also seems to be the format expected by Facebook. A nice feature would be to have an option to invert the image luminance.

Hi Jeffrey! Didn't see your comment until now. Will definitely look into this as soon as I get some time over (unless you want to submit a pull request of course).
A quick thought for now is that since the MPFExtractor doesn't actually modify any images, it might make sense to build that as a separate module.


Support latest build packages, migrate to null safety

Support the latest version of the build system and migrate to the implementation to null safety.

Sorry if I'm bothering you here @nshahan @luisvt, but could you maybe take a look at this? I think it would be great if we could use this package with the latest version of the build system.


How to specify the size of a pool to be able to quiesce it to zero ?

For testing purposes and to save on cost (especially in testing environments) we can nowadays use serverless databases:
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-serverless#autopausing-and-autoresuming
To be able to use it number of sessions has to drop to 0.
How to specify the hikari pool size to reduce number of connections to zero (when app is idle)  ?
Of course in this case we sacrifice performance for cost but let's assume we focus on cost.
Is setting minimumIdle to 0 enough to achieve it ?

You can achieve it with ds.setMinimumIdle(0), please note the default idleTimeout is 10 minutes, you should shorten it.