Add html-minifier

Please make this optional and disabled by default. When enabled, I think it should accept the options object so that a user may enable/disable certain options (in particular, removeComments: true is not desired in some applications). true may mean everything enabled.

Not sure if this is safe to do, it might break Client Side Hydration

The tests now fail on Linux because you forgot to change the module file name casing.
Otherwise looks good to me.

Sorry, vscode doesn't recognize renaming when change case of filename.

Now it will be ok.

:tada: This issue has been resolved in version 4.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

I published this with a few changes. From the user point of view, I renamed the option to minifyHtml (which goes in better accordance with minimize).
In the code, I first uniformed the HTML rendering so that minifyHtml could be applied in one place instead of copy/pasting the logic.


how train LORA with multiple concepts/subjects?

Please authors tell me how I can train LORA with multiple concepts/subjects?

its not possible using autotrain at this moment


PFUser does not have a member named subscript

Hello,
I seem to just be having an awful time when it comes to errors and parse. So today i was cleaning out my downloads folder and i deleted the parse ios sdk unzipped folder. This in turn, caused some errors in my xcode project. I redownloaded the sdk and added it back to my xcode project which shouldve solved it but now i have like 50+ errors. Each error says "PFUser does not have a member named subscript". My whole project was running perfectly yesterday so i know its not just the syntax or anything like that. The only help ive seen onine is to change the syntax but thats a pain to do it for 50 lines of code. It might be helpful to note that when i try to move the parse frameworks into my frameworks folder, all the frameworks turn red. But when they are sitting outside the frameworks folder everything is fine. Any advice? It sucks because my app is already on the app store and i have some important updates i want to make and i cant do anything because the build fails every time.

Ended up solving the issue by rewriting the syntax for each error. Took a while but everything seems to be working fine

Just for reference, the issue surfaces only if you are using Xcode 6.4. Since Xcode 7 was already released and available - I would strongly recommend updating to the latest Xcode.


v5.28 Changelog

@mkraft For v5.28 changelog, can you help with the following information:

New config.json settings
Database Changes
API Changes
Websocket Event Changes


Customized setting for ping-pong trade (Sample: BTC-XRP pair in Poloniex)

Hi, this is a sample of customized setting I made to make ping-pong trade. I'm using it to trade BTC-XRP pair in Poloniex.
1. I'm using short periode RSI to generate BUY-SELL signals. (inside config.js):
config.tradingAdvisor = {   enabled: true,   method: 'RSI',   candleSize: 1,   historySize: 3,   adapter: 'sqlite',   talib: {     enabled: false,     version: '1.0.2'   } }
// RSI settings: config.RSI = {   interval: 3,   thresholds: {     low: 30,     high: 70,     // How many candle intervals should a trend persist     // before we consider it real?     persistence: 1   } };
2. Inactivate cancel orders step. Just to let gekko execute all orders, and wait the pending orders to be executed later whenever the price movement strikes on them. (inside potfoliomanager.js):
`Manager.prototype.checkOrder = function() {
var handleCheckResult = function(err, filled) {
if(!filled) {
//log.info(this.action, 'order was not (fully) filled, cancelling and creating new order');
//-----inactivate below line to skip cancel orders
//this.exchange.cancelOrder(_.last(this.orders), _.bind(handleCancelResult, this));
  return;
}

`
3. Customize the BUY-SELL setting (inside potfolioManager.js, in the "Manager.prototype.trade" function):
`this.action = what;
var act = function() {
var amounttobuy, amounttosell, price, upperprice, lowerprice, minimumbalance, minimumasset, lotsize;
//if signal BUY appears
if(what === 'BUY') {

  //Split your bankroll at least into 10 lots. Lotsize is in currency (BTC).
  lotsize = 0.00010500;

  //define price to sell HI, 7 satoshi above current bid to make quick sell
  price = this.ticker.bid + 0.00000007;
  amounttosell = (lotsize + (0.005*lotsize)) / price;

  //define price to buy LO, 0.5% below sell price
  lowerprice = price - (0.005 * price);
  amounttobuy = lotsize / lowerprice;

  //define minimum parameter to execute pingpong trade
  minimumasset = amounttosell + (0.01*amounttosell);
  minimumbalance = 0.00011000;

  //execute buy LO if balance > minimum parameter
  if(this.getBalance(this.currency) > minimumbalance){
      this.buy(amounttobuy, lowerprice);
  }

  //execute sell HI if asset > minimum parameter
  if(this.getBalance(this.asset)>minimumasset){
		  this.sell(amounttosell, price);
  }

//if signal SELL appears
} else if(what === 'SELL') {

  //Split your bankroll at least into 10 lots. Lotsize is in currency (BTC).
  lotsize = 0.00010500;

  //define price to buy LO, 7 satoshi below current ask to make quick buy
  price = this.ticker.ask - 0.00000007;
  amounttobuy = lotsize / price;

  //define price to sell HI, 0.5% above buy price
  upperprice = price + (0.005*price);
  amounttosell = (lotsize + (0.005*lotsize)) / upperprice;

  //define minimum parameter to execute pingpong trade
  minimumbalance = 0.00011000;
  minimumasset = amounttosell + (0.01*amounttosell);

  //execute sell HI if asset > minimum parameter
  if(this.getBalance(this.asset) > minimumasset){
      this.sell(amounttosell, upperprice);
  }

  //execute buy LO if balance > minimum parameter
  if(this.getBalance(this.currency) > minimumbalance){
	  this.buy(amounttobuy, price);
  }
}`

Please have a try, and you might give a feedback to furnish the setting.
Note: not sure if the setting can be backtested. I'm only using it in real trade mode thru terminal and checking the result in polo trade history.

Thanks a lot for sharing! When I am working on exchange integration and I need to test buys and sells I use this strategy which triggers a signal every 3 minutes (BUY/SELL/BUY/..): https://gist.github.com/askmike/959cada3c68f0bb3904ef08e7db28073

I tried a similar approach, ie: a time-based pingpong strategy eg.: every 2, 3 or 5 minutes strikes. (I used the strategy in CoinExchange by created a js+iMacros script). Basically it's working nicely in sideways market, but it's gonna eat up our bankroll during trending market when the price goes to opposite direction than we expect.
Since gekko has integrated TA strategies, I tried to utilize it as a trigger to determine the correct time to execute buy-sell order. I've been running this approach for 2 days, and the balance is stable after >200 strikes. What I need to do is furnishing the approach by making some tweaks, either on the RSI setting, or using different TA strategy other than RSI, or making lot size management to cover market fee (which is 0.15% in Poloniex).
I'm pretty sure that the above setting I made will give nice result if we can run it in 0% market fee exchange, such as in vipbitcoin exchange which I requested @danKHUU to add it into gekko in 1186.

If you are looking for a 0 fee exchange for market makers, you can use GDAX
right now :)

Since gekko has integrated TA strategies, I tried to utilize it as a
trigger to determine the correct time to execute buy-sell order.

That's a lot better than mine, though I only use it to debug and program on
Gekko.
If you put your strategy in a gist (and how you think this works best, eg.
small candles on highly liquid pairs) I can add it to this page:
https://gekko.wizb.it/docs/extending/other_software.html
You can create a gist here: https://gist.github.com/new
On Sat, Oct 28, 2017 at 10:01 AM, bitkominer<EMAIL_ADDRESS>wrote:

I tried a similar approach, ie: a time-based pingpong strategy eg.: every
2, 3 or 5 minutes strikes. (I used the strategy in CoinExchange by created
a js+iMacros script). Basically it's working nicely in sideways market, but
it's gonna eat up our bankroll during trending market when the price goes
to opposite direction than we expect.
Since gekko has integrated TA strategies, I tried to utilize it as a
trigger to determine the correct time to execute buy-sell order. I've been
running this approach for 2 days, and the balance is stable after >200
strikes. What I need to do is furnishing the approach by making some
tweaks, either on the RSI setting, or using different TA strategy other
than RSI, or making lot size management to cover market fee (which is 0.15%
in Poloniex).
I'm pretty sure that the above setting I made will give nice result if we
can run it in 0% market fee exchange, such as in vipbitcoin exchange which
I requested @danKHUU https://github.com/dankhuu to add it into gekko in
1186 https://github.com/askmike/gekko/issues/1186.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/askmike/gekko/issues/1252#issuecomment-340134883, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA7MDyHk4iiQ92cuVPddiwyG41S_Ih4fks5swpjtgaJpZM4QJw1o
.

--
PGP key at keybase.io/mikevanrossum
https://keybase.io/mikevanrossum/key.asc

Great. I'll take a look on it first.

If you are looking for a 0 fee exchange for market makers, you can use GDAX right now :)

Well... the smallest order of 0.01 in GDAX is quite big, especially for "micro" traders like me :)
That's why currently Poloniex is my favorite until @danKHUU comes back with vipbitcoin integration.

The above pingpong strategy can be downloaded from here: portfolioManager_pingpong.
It contains 2 supplemental js files to be saved inside folder: /plugins/trader/
To change back to default gekko portfolioManager, just inactivate pingpong variable inside trader.js (line#11):
var activatepingpong = false;

Ah in order for your strategy to work you also needed to patch certain files. What you can also do is create a new fork of Gekko that includes all your changes, that makes it easier for other users to run your version.

It was actually optional to patch certain files, since it can be done also inside portfolioManager individually. But it was quite complicated for some users.
I've never done creating any fork before, but I'll try to do it. Github is a new thing for me... :)
Hope I can do it soon.
Thanks!

Hello @bitkominer and @askmike ,
Can you please give some advice to set this up?
I tried to set up the ping pong trader but now I still have to patch the files like askmike notices.
Any help on this? Now I get the error that I can not get market data.
Thank you.

Hi,
when i try to run the portfolioManager_pingpong with commandline live gekko
i receive following error, could you please let me know how to patch files?
Thx in advance,
`2018-03-05 21:22:30 (INFO):	Setting up:
2018-03-05 21:22:30 (INFO):		 Advice logger
2018-03-05 21:22:30 (INFO):
2018-03-05 21:22:30 (INFO):	Setting up:
2018-03-05 21:22:30 (INFO):		 Trader
2018-03-05 21:22:30 (INFO):		 Follows the advice and create real orders.
2018-03-05 21:22:30 (DEBUG):	getting ticker, balance & fee from binance
/gekko/plugins/trader/trader.js:13
this.manager.init(next);
^
TypeError: this.manager.init is not a function
at gekko/plugins/trader/trader.js:13:16)
at /gekko2/gekko/core/pluginUtil.js:95:22)
at /gekko2/gekko/node_modules/async/dist/async.js:1156:9
at /gekko2/gekko/node_modules/async/dist/async.js:1030:17)
at /gekko2/gekko/node_modules/async/dist/async.js:1015:17)
at /gekko/node_modules/async/dist/async.js:988:16
at /gekko/node_modules/async/dist/async.js:1158:13
at /gekko/core/pluginUtil.js:103:9
at Timeout._onTimeout (/gekko/node_modules/lodash/dist/lodash.js:5496:43)
at ontimeout (timers.js:458:11)
`

Hello guys. Many thanks for you efforts on gekko.
Is there any chance to backtest the modified portfolio manager from @bitkominer? As far as I understand the architecture, it should be hard to get meaningful results. We don't know if one or both orders will be filled or not. During backtest we can't answer that for sure, because we don't have access to the order book. Please correct me if I'm wrong. Do you really test this out with real money until it works? 🤔

have the same issue as @sterns - was attempting to edit the pingpong files for my own but realise even basic use of them fails (after rolling back my edits).
Any advice as to why this wouldn't work, maybe the new updates are clashing with this older script?
Any help greatly appreciated


Go Ethereum allows attackers to use manipulation of time-difference values to achieve replacement of main-chain blocks

Summary of Bug
Go Ethereum (aka geth) through 1.10.21 allows attackers to increase rewards by mining blocks in certain situations, and using a manipulation of time-difference values to achieve replacement of main-chain blocks, aka Riskless Uncle Making (RUM), as exploited in the wild in 2020 through 2022.

Expected Behaviour
CVE-2022-37450
GHSA-rqmg-hrg4-fm69

For Admin Use

[ ] Not duplicate issue
[x] Appropriate labels applied
[x] Appropriate contributors tagged/assigned

@mik-patient Are you sure this issue belongs to here?

Hi @crodriguezvega Thank's for your respond, and bellow the more information about vulnerabilities at our projects.
Additional Informations
ibc-go
Go Ethereum (aka geth) through 1.10.21 allows attackers to increase rewards by mining blocks in certain situations, and using a manipulation of time-difference values to achieve replacement of main-chain blocks, aka Riskless Uncle Making (RUM), as exploited in the wild in 2020 through 2022.
Now we found at
github.com/ethereum/go-ethereum v1.10.17 h1:XEcumY+qSr1cZQaWsQs5Kck3FHB0V2RiMHPdTBJ+oT8=
github.com/ethereum/go-ethereum v1.10.17/go.mod h1:Lt5WzjM07XlXc95YzrhosmR4J9Ahd6X2wyEV2SvGhk0=

Ah, sorry, I read the issue then the wrong way and didn't realize that you were referring to a dependency...

Since go-ethereum is an indirect dependency, is there anything we can do in ibc-go to update it to the latest? or we just need to wait until our direct dependencies that use it update?

Hi! Team @crodriguezvega
Thank you for your respond. I hope you will be happyness days.
Additional Information (for fixed):
There is no fixed version for github.com/ethereum/go-ethereum/core.
if you don't use the ethereum package, you can remove it for fixed.


reuse thread pool after calling stop(true) ?

Hi,
If I try to reuse the thread pool (using push) after calling stop(true), it doesn't create any new threads ? Something like the following sequence:
auto thread_pool  = new ctpl::thread_pool(2);
for (int i = 0; i < 10; i++)
{
    thread_pool->push(func, i);
}
thread_pool->stop(true);
thread_pool->resize(4);
/* Wait a while */
thread_pool->push(func, i); // ==> func is not executed

Is this behavior intentional ? I have gotten around this by destroying the old pool and creating a new pool but that does not seem efficient.

I have the same question. Or to rephrase it:
Is there any way to call "join" to all functions in the pool and queue without deleting the threads?

From reading over the source, the simplest fix would be to add a start() function like this
void start(int nThreads){
    this->init();
    this->resize(nThreads);
}

This effectively emulates the behavior of the constructor:
    `thread_pool(int nThreads, int queueSize = _ctplThreadPoolLength_) : q(queueSize) { this->init(); this->resize(nThreads); }`

Which would allow you to restart the pool by effectively rebuilding the threads.
While I feel this would work, it would be a cheap fix to the underlying problem.
In a more ideal implementation the resize() function would be written in a manor that would allow the resize to be performed without stopping all work. This would involve synchronizing the changes to the thread container with the workers themselves, allowing the workers to pause working instead of going to a full stop while threads are being added or removed.

Does it delete and create all threads again?
Ideally we do not want unnecessary system calls to create threads.

The bit of code I provided is the naive approach to solving the problem. It would wind up having to delete and rebuild the threads. See the last paragraph of my original comment.
A better solution would be a system that that can pause work on all of the threads and only kill the ones you that are needed. The issue isn't with the functionality of stop(). The issue is what it is being used for. It is being used before calling resize() to allow safe resizing of the pool.
It would be better to have another function that could pause work or have resize be written in a way that it can resize without having to stop work at all.


crd-schema-check: allow new CRDs

If the 'git show BASE_REF' fails assume it is a net new CRD and should be fine.

@stuggi: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
Test name 	Commit 	Details 	Required 	Rerun command
ci/prow/glance-operator-build-deploy-kuttl 	7504281 link 	true 	/test glance-operator-build-deploy-kuttl
Full PR test history. Your PR dashboard.

{  error occurred handling build src-amd64: could not get build src-amd64: builds.build.openshift.io "src-amd64" not found}

/retest


Data Tiling: Folding tensor.pack(tensor.unpack) to simplified form

If they have the same attributes and there are no padding values, we can just fold them away. They become the source of unpack op.
If they have different attributes and there are no padding values, we should be able to decompose the ops and they will become transpose + reshapes + transpose ops. The transpose op can be fused with its producers and consumers; the reshape ops are just metadata ops. E.g.,
%unpack_3762 = tensor.unpack %1462
  inner_dims_pos = [0, 1]
  inner_tiles = [16, 16]
  into %30 : tensor<24x8x16x16xf32> -> tensor<384x128xf32>
%pack_3763 = tensor.pack %unpack_3762
  inner_dims_pos = [0, 1]
  inner_tiles = [16, 1]
  into %37 : tensor<384x128xf32> -> tensor<24x128x16x1xf32>

After decomposition:
%transposed = linalg.transpose
  ins(%arg0 : tensor<24x8x16x16xf32>)
  outs(%1 : tensor<24x16x8x16xf32>)
  permutation = [0, 2, 1, 3]
%collapsed = tensor.collapse_shape %transposed [[0, 1], [2, 3]]
  : tensor<24x16x8x16xf32> into tensor<384x128xf32>
%expanded = tensor.expand_shape %collapsed [[0, 1], [2, 3]]
  : tensor<384x128xf32> into tensor<24x16x128x1xf32>
%transposed_1 = linalg.transpose
  ins(%expanded : tensor<24x16x128x1xf32>)
  outs(%2 : tensor<24x128x16x1xf32>)
  permutation = [0, 2, 1, 3]

One questions is that if we want to do it before forming dispatches. The reshape ops become fusion barrier in this case and we can fuse them with consumers and producers. W/o decomposition, we can form unpack + pack into a dispatch; we can tile and distribute the work. IMO, it's a bit worse because we need an extra kernel launch (if there are producers and consumers).

The simplification will result in more dispatch launches because the reshape op becomes a barrier. We should just fuse unfoldable unpack+pack into a dispatch and codegen.


Fix Postgres array syntax for UPDATE

This PR should address https://github.com/kayak/pypika/issues/579 by formatting UPDATE queries using the same rules as INSERT queries. Regression tests for the new case have been added, courtesy of @vfonte91.
A side effect of this fix is that I discovered PyPika has inconsistent handling of null versus NULL. In these queries, null is used:
SELECT * FROM "abc" WHERE "foo"=null
UPDATE "abc" SET "foo"=null

But in these queries, NULL is used:
INSERT INTO "abc" VALUES (NULL)
INSERT INTO "abc" VALUES (1) RETURNING NULL
SELECT NULL "abcdef"

This is because there are two different code paths for formatting null values:
https://github.com/kayak/pypika/blob/a7b01dac790f4c6e93a961b0be26a9645fe7cf5c/pypika/terms.py#L385
https://github.com/kayak/pypika/blob/a7b01dac790f4c6e93a961b0be26a9645fe7cf5c/pypika/terms.py#L479
Both of these code paths used to return lowercase null circa 2018; this was changed in a seemingly unrelated commit https://github.com/kayak/pypika/commit/3bfee98bb226d4736b03473137e45e4d3e78a127.
Some other keywords (true, false) seem to be lowercase by convention in PyPika, whereas others (SYSTEM_TIME) seem to be uppercase by convention, so it's not entirely clear to me what the expected standardization is. On the plus side, since SQL is fully case-insensitive, it doesn't really matter too much.
In this PR, I've updated all code paths and tests to use lowercase null. Let me know if that's wrong.

would love this to be merged :(

@ Maintainers - any chance of getting this merged?


[semantic-arc-opts] Use the LiveRange abstraction to find destroys instead of iterating over direct uses so we handle forwarding uses.

Previously, we were incorrectly handling these test cases since we weren't
properly finding destroys through forwarding instructions. I fixed that in a
previous commit by changing the code here to only support load [copy] without
forwarding instructions.
In this commit, I change the code to instead use the LiveRange abstraction. The
LiveRange abstraction already knows how to find destroys through forwarding
instructions and has this destroy array already computed for us!
So we get better runtime performance of code (due to the better opt) and better
compile time (since we aren't computing the destroy list twice)!
rdar://58289320

@swift-ci test

@swift-ci benchmark

Build failed
Swift Test OS X Platform
Git Sha - 2b09547c0713f91c1e3a999f0d809f80104d63ed

@swift-ci test os x platform

Performance: -O

Improvement
OLD
NEW
DELTA
RATIO

FlattenListFlatMap
4876
4440
-8.9%
1.10x (?)

Code size: -O
Performance: -Osize

Improvement
OLD
NEW
DELTA
RATIO

FlattenListLoop
2756
2572
-6.7%
1.07x (?)

Code size: -Osize
Performance: -Onone

Improvement
OLD
NEW
DELTA
RATIO

DataAppendDataSmallToSmall
3200
2800
-12.5%
1.14x (?)

Code size: -swiftlibs

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

No regressions, good. I figured we wouldn't see much since we hadn't hit this bug yet.

Build failed
Swift Test Linux Platform
Git Sha - 2b09547c0713f91c1e3a999f0d809f80104d63ed

@swift-ci test linux platform

@swift-ci test windows platform


Certificate Chain - Adding/creating a validator as well as documenting about them

Certificate Chains aren't well understood by customers.
Rancher agent uses the full chain to connect to rancher server.
Add learning content. Review existing stuff about cert chains. Describe that you have to pass down the construction of the entire chain.

@divya-mohan0209 going over past issues -- did this draft ever make it into the docs?


All for alternative policy baselines

This PR updates OG-USA to allow one to specify the baseline individual income tax policy so that alternative IIT baseline (i.e., alternatives to the Tax-Calculator current law baseline) can be used in OG-USA.
An example script that implements a current policy baseline is added to illustrate usage.

Codecov Report
Attention: Patch coverage is 88.88889% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 75.35%. Comparing base (e4655db) to head (48214b1).
Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #106      +/-   ##
==========================================
- Coverage   75.41%   75.35%   -0.06%
==========================================
  Files          11       11
  Lines         850      852       +2
==========================================
+ Hits          641      642       +1
- Misses        209      210       +1

Flag
Coverage Δ

unittests
75.35% <88.88%> (-0.06%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

ogusa/calibrate.py
37.41% <ø> (ø)

ogusa/get_micro_data.py
96.82% <88.88%> (-1.54%)
:arrow_down:


Exercise 4.10 How many dead bodies in johnDoe have age upper limit (年齡範圍上限) equal to 0?

Exercise 4.10 How many dead bodies in johnDoe have age upper limit (年齡範圍上限) equal to 0?

https://github.com/raychiu135/110-1-r4ds-main/blob/314ccf3565b59a4c44848610427731d349ffef65/exercise_4.10.rmd#L2

https://github.com/linttttt/110-1-r4ds-main/blob/63b866865f5c68596af5a38200bd4acdb4369b3d/studyCH4(EX).Rmd#L78

https://github.com/32119811/110-1-r4ds-main/blob/815c3b5cd869030c6f7d38cde3a046336355ec1e/Exercise 4.10#L3

https://github.com/32119811/110-1-r4ds-main/blob/815c3b5cd869030c6f7d38cde3a046336355ec1e/Exercise 4.10#L3

@32119811  comparing vectors of different classes is very dangerous.

https://github.com/AmyChen20021003/110-1-r4ds-main/blob/1ed15acbf583e52352b7d7fb7ed93a585f488683/4-9.Rmd#L21

https://github.com/Vincent23412/110-1-r4ds-main/blob/22fd3b28e9ff751adbf7f00a6e7b3f37c60d14d0/20211212.Rmd#L19


Cannot read properties of undefined (reading 'find) when tried to chat with a created kB.

Logs:
Creating embeddings for Ollama served model: nomic-embed-text:latest
Creating Chroma vector store
Initializing ParentDocumentRetriever with RedisDocstore
Redis client options:  { host: 'redis', port: 6379, username: undefined, password: undefined }
19 URLs added to collection collection_17.
All URLs: Set(19) {
'https://vmc.techzone.vmware.com/resource/vmware-cloud-aws-network-architecture',
'https://vmc.techzone.vmware.com',
'https://vmc.techzone.vmware.com/vmware-cloud-well-architected#framework',
'https://vmc.techzone.vmware.com/vmware-cloud-well-architected',
'https://vmc.techzone.vmware.com/vmware-cloud-well-architected#designs',
'https://vmc.techzone.vmware.com/alibaba-cloud-vmware-service',
'https://vmc.techzone.vmware.com/azure-vmware-solution',
'https://vmc.techzone.vmware.com/google-cloud-vmware-engine',
'https://vmc.techzone.vmware.com/oracle-cloud-vmware-solution',
'https://vmc.techzone.vmware.com/vmware-cloud-aws',
'https://vmc.techzone.vmware.com/vmware-cloud-aws-outposts',
'https://vmc.techzone.vmware.com/multi-cloud',
'https://vmc.techzone.vmware.com/vmware-sovereign-cloud',
'https://vmc.techzone.vmware.com/app-modernization',
'https://vmc.techzone.vmware.com/migration',
'https://vmc.techzone.vmware.com/cloud-vdi',
'https://vmc.techzone.vmware.com/data-center-extension',
'https://vmc.techzone.vmware.com/disaster-recovery',
'https://vmc.techzone.vmware.com/blog'
}
Knowledge base file created with ID:  15
Knowledge base file created with ID:  16
Knowledge base file created with ID:  17
Knowledge base file created with ID:  18
Knowledge base file created with ID:  19
Knowledge base file created with ID:  20
Knowledge base file created with ID:  21
Knowledge base file created with ID:  22
Knowledge base file created with ID:  23
Knowledge base file created with ID:  24
Knowledge base file created with ID:  25
Knowledge base file created with ID:  26
Knowledge base file created with ID:  27
Knowledge base file created with ID:  28
Knowledge base file created with ID:  29
Knowledge base file created with ID:  30
Knowledge base file created with ID:  31
Knowledge base file created with ID:  32
Knowledge base file created with ID:  33
URL: /api/knowledgebases User: {"id":2,"name":"wukong919","email":"wukong919@hotmail.com","role":"user","scope":["test","user"],"iat":1726710901,"exp":1758246901}
URL: /api/instruction User: {"id":2,"name":"wukong919","email":"wukong919@hotmail.com","role":"user","scope":["test","user"],"iat":1726710901,"exp":1758246901}
URL: /api/knowledgebases User: {"id":2,"name":"wukong919","email":"wukong919@hotmail.com","role":"user","scope":["test","user"],"iat":1726710901,"exp":1758246901}
URL: /api/models/chat User: null
Chat with knowledge base with id:  17
Knowledge base test1000 with embedding "nomic-embed-text:latest"
Creating embeddings for Ollama served model: nomic-embed-text:latest
Creating Chroma vector store
Initializing ParentDocumentRetriever with RedisDocstore
Redis client options:  { host: 'redis', port: 6379, username: undefined, password: undefined }
[nuxt] [request error] [unhandled] [500] Cannot read properties of undefined (reading 'find')
at createChatModel (./.output/server/chunks/_/index2.mjs:453:35)
at Object.handler (./.output/server/chunks/routes/api/models/index.post.mjs:182:18)
at async Object.handler (./.output/server/chunks/runtime.mjs:3126:19)
at async Server.toNodeHandle (./.output/server/chunks/runtime.mjs:3392:7)
Any clue of what did cause the issue?

cant reproduce the issue any more.


Unable to generate API summary when access key is provisioned by Develocity plugin/extension

The API summary currently only looks for API keys in enterprise/keys.properties. As of Develocity Gradle Plugin 3.17, keys are generated and stored in develocity/keys.properties: https://docs.gradle.com/enterprise/gradle-plugin/current/#automated_access_key_provisioning
This means that users who provision a key with Develocity Gradle Plugin 3.17+ will not be able to properly generate the summary at the end of the experiments.
I assume the same is true for Maven as well.

I am experiencing something related if you do not define the API token at all. In my case, the build scans can be published but the summary cannot be produced due to permission issues. I'd suggest adding an initial verification call to the Develocity server that sends the token to check if it is valid. Only then proceed with generating the first build scan.

For now we recommend you to use the older versions of the Gradle Enterprise Gradle plugin 3.16.2 and Gradle Enterprise Maven extension 1.20.1 to run the Build Validation Scripts.
You can use the latest versions of the Develocity Gradle plugin and Develocity Maven extension for all other use cases.
Note that if Publish build scans is not granted anonymously, then an access key granting the permission is required in at least one of the following places when using these older plugin/extension versions:
For Gradle:

enterprise/keys.properties in the Gradle user home directory
GRADLE_ENTERPRISE_ACCESS_KEY environment variable

For Maven:

.gradle-enterprise/keys.properties in the Maven user home directory
GRADLE_ENTERPRISE_ACCESS_KEY environment variable

If Access build data via the API is not granted anonymously and the Performance Characteristics section of the summary is desired, or in automation scenarios, then an access key granting the permission is required in at least one of any of the above mentioned locations.
In order to verify connectivity and that everything works as expected, you can run the following command:
For Gradle:
./01-validate-incremental-building.sh -t assemble -r https://github.com/gradle/gradle-build-scan-quickstart -e -s https://develocity.example.io

For Maven:
./01-validate-local-build-caching-same-location.sh -g compile -r https://github.com/gradle/maven-build-scan-quickstart -e -s https://develocity.example.io

This was resolved in version 2.6 of the Develocity Build Validation Scripts. Since that version, the following locations are considered when searching for an access key, where 1 is considered first and 6 is considered last:

The DEVELOCITY_ACCESS_KEY environment variable
The GRADLE_ENTERPRISE_ACCESS_KEY environment variable
The ~/.gradle/develocity/keys.properties file
The ~/.gradle/enterprise/keys.properties file
The ~/.m2/.develocity/keys.properties file
The ~/.m2/.gradle-enterprise/keys.properties file

If no access key is found for the configured Develocity server, then the scripts will attempt to call the API anonymously.
Closing this issue.


#WIP added summary to about to improve preview

fixes #13
@beljun yes we can close 13 but also will use this PR to fix the summary on about and then close it.

Something seems off @beljun this wont render locally

@beljun I guess we cannot use the heading and summary in the about page


Update TZ build instructions

Updated the TrustZone build instructions to reflect the use of tz_env.sh rather than manually sourceing files.

+1 reached, merging.


Unable to access lidar data

In the examples section there is no mention on how to use the lidar data. Is there any way to access it using only python sdk without ros ?

it would be great to have this feature


Topic needed: Contributing - Katacoda

Guide for contributors who wish to create a Katacoda live tutorial

When to create a live tutorial
Types of live tutorials
Prerequisites
Identifying your audience
Create the outcome
Identifying steps
Detail the work
Test the steps
Verifying steps
Embedding the demo

Partially-completed in #74 - needs some details fleshed out, questions answered


Create an Example for "Create File" functionality

🆕🐥 First Timers Only
This issue is reserved for people who have never contributed to Hedera or any open source project in general.
We know that creating a pull request (PR) is a major barrier for new contributors.
The goal of this issue and all other issues labeled by 'good first issue' is to help you make your first contribution to Hedera.
👾 Description of the issue
Problem
The Hedera Java SDK includes a simple but essential example, CreateFileExample.java, which demonstrates how to create, manage, and delete a file on the Hedera network. This example showcases a key functionality of the Hedera File Service, helping developers understand how to interact with files on Hedera.
However, there is no equivalent example available in the Hedera JavaScript SDK. Developers using the JavaScript SDK would benefit from a similar example that guides them through the process of creating a file, setting the file's contents, and managing the file lifecycle (including deletion).
Solution
Below is a link to the existing Java implementation, which can serve as a basis for creating the JavaScript example:
CreateFileExample.java
Steps to Implement

Client and operator setup: Set up the Hedera Client in the JavaScript SDK using setOperator.
File creation:

Use FileCreateTransaction to create a new file, specifying the operator's public key as the owner.
Set the file contents (e.g., a string or byte array).
Set an appropriate transaction fee (e.g., 2 Hbars).

File deletion: Demonstrate how to delete the created file using FileDeleteTransaction.
Logging and output: Include console logs to show the progress of file creation, retrieval of the file ID, and deletion for better user understanding.

Expected Outcome
The new example will help developers using the JavaScript SDK understand how to create, manage, and delete files using the Hedera File Service. It will also highlight key SDK features such as handling file contents, associating keys with files, and managing transaction fees.
By providing this JavaScript example, we align the JavaScript SDK with the Java SDK in terms of documentation and usability, making it easier for developers to build Hedera-based applications.
📋 Step by step guide to do a contribution
If you have never contributed to an open source project at GitHub, the following step-by-step guide will introduce you to the workflow. More information and concrete samples for shell commands for each step can be found in our CONTRIBUTING.md file.
A more detailed general documentation of the GitHub PR workflow can be found here.

[ ] Claim this issue: Comment below that you are interested in working on the issue
[ ] Wait for assignment: A community member with the given rights will add you as an assignee of the issue
[ ] Fork the repository: You can do that in GitHub (by simply clicking the 'fork' button).
[ ] Check out the forked repository
[ ] Create a feature branch for the issue. We do not have a hard naming definition for branches but it is best practice to prefix the branch name with the issue id.
[ ] Solve the issue in your branch.
[ ] Commit your changes: Here, it is needed to add sign-off information to the commit to accept the "Developer Certificate of Origin" (https://developercertificate.org). More details can be found in our CONTRIBUTING.md
[ ] Start a Pull Request (PR): We have a pattern for naming pull requests that a GitHub Action checks. We use that pattern to support the creation of automatic release notes.
[ ] Check GitHub Actions: Several GitHub Actions will be triggered automatically for each PR. If a GitHub Action fails and you do not understand the cause of that error do not hesitate to add a comment to the PR and ask the Hedera developer community for support.
[ ] Wait for reviews: Members of the Hedera developer community will review your PR. If a reviewer finds any missing pieces or a problem, he or she will start a discussion with you and describe the next steps for solving the problem.
[ ] You did it 🎉: We will merge the fix in the develop branch. Thanks for being part of the Hedera community as an open-source contributor ❤

🎉 Contribute to Hacktoberfest
Solve this issue as part of the Hacktoberfest event and get a chance to receive cool goodies like a T-Shirt. 🎽
🤔 Additional Information
If you have any questions, just ask us directly in this issue by adding a comment. You can join our community chat at Discord. A general manual about open-source contributions can be found here.

Hi @hendrikebbers ,
I would love to work on this.
Can you please assign it to me?

Hello @hendrikebbers please assign me this issue if it's still hasn't been worked on

@vickywane are you working on the issue? Otherwise I will assign it to @b-l-u-e

Hello @hendrikebbers
I am working on this issue.
I will have a PR up for it really soon.

Is this done? @hendrikebbers


Geometric Progression in DART

Have you read the Contributing Guidelines on Pull Requests?
Yes
Description
The dart script determines if the array consisting a sequence of elements is in Geometric Progression or not.
Checklist

[X] I've read the contribution guidelines.
[X] I've checked the issue list before deciding what to submit.
[X] I've edited the README.md and link to my code.

Related Issues or Pull Requests
Fixes #7053

@all-contributors bot please add @RohiniRG for code


Installation

Thanks for the amazing product.
I have few queries:
How to make it analysis for every app , itseems to be profiling only for the AndroidGodEye App.
Also how to install in windows , i tried following the instructions but no help.

How to make it analysis for every app , itseems to be profiling only for the AndroidGodEye App.
Please follow the instruction of wiki, it need to be integrated to your app
https://github.com/Kyson/AndroidGodEye/wiki/0x00-QuickStart_en#quickstart
Also how to install in windows , i tried following the instructions but no help.
Generally you can install the AndroidGodEye Android Studio Plugin, if not, try Windows bash script: AndroidGodEye_plugin.bat


Add StormyLaunch by @Sendeky

[x] Project URL:
[x] Update contents.json instead of README
[x] One project per pull request
[x] Screenshot included
[x] Avoid iOS or open-source in description as it is assumed
[x] Use this commit title format if applicable: Add app-name by @github-username
[x] Use approved format for your entry

Some formatting issues but I will fix


Update guidance site for changes to component sizing from default to medium

Summary
Once changes for  https://github.com/mi6/ic-ui-kit/issues/2072 are complete, the site should be reviewed to ensure the examples use medium instead of default
Following components have had size 'default' updated to 'medium':
Accordion
Badge
Button
Checkbox
Chip
Date picker
Empty state
Loading indicator
Page header
Radio button
Search bar
Status tag
Switch
Text field
Top navigation

Blocked by releasing v3


Remove telemetry generation from identity server, will not be used.

Removes telemetry definition generation added in https://github.com/aws/language-servers/pull/625 This creates coupling to toolkit-specific telemetry and brings in much more than is needed.  As aws-lsp-identity is a Flare server and defined in Flare, the limited telemetry emitted by this server will be defined directly in this repo as well.

the limited telemetry emitted by this server will be defined directly in this repo as well.

If it's limited, then only the limited metrics from toolkits telemetry would be used.

This creates coupling to toolkit-specific telemetry and brings in much more than is needed.

Metrics in https://github.com/aws/aws-toolkit-common are "common". Any toolkit-specific metrics can just be not used by clients that don't want them. But for metrics that are useful, isn't it better to align on common shapes?

the limited telemetry emitted by this server will be defined directly in this repo as well.

If it's limited, then only the limited metrics from toolkits telemetry would be used.

This creates coupling to toolkit-specific telemetry and brings in much more than is needed.

Metrics in https://github.com/aws/aws-toolkit-common are "common". Any toolkit-specific metrics can just be not used by clients that don't want them. But for metrics that are useful, isn't it better to align on common shapes?

The identity server in Flare is available to all clients ("destinations") that consume Flare, not just AWS Toolkits.  Metrics generated in the identity server are emitted to clients (not posted to backends) via the LSP telemetry/event notification and it is up to the client to handle that metric as it sees fit (whether this is to ignore, transform, and/or post it to some backend system).
There is a possibility TKC may post error/usage related metrics specific to the identity server's operation itself for support purposes.  That implementation can be designed as the need arises.


Mouse Over Information

Is your feature request related to a problem? Please describe.
-No
Describe the solution you'd like

If the mouse hovers over Text which is grey then additional information why i can not click on the text would be nice

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots or a mockup about the feature request here.

My Programs and save are unavailable because you might have not logged in.
NEPO-Blocks beginner is unavailable because you have selected Expert one.


spec: Rename app name to image name.

Use app name in this section seems a little bit out of tune to me. How about changing it to image name?
/cc @jonboulle @philips

I think by and large this makes sense. One critical omission here: the "apps" field in the "dependencies" list.

Updated.

@yifan-gu sorry, while you're in there..

@jonboulle NW. Updated again.

@yifan-gu Spec change lgtm. Please update the schema alongside this, sorry I forgot to mention that earlier...

@jonboulle Then I found we need to change a bunch of places as well.... I have changed all the appearance of ImageID.  PTAL.

/cc @krobertson any thoughts on this?

LGTM, make more sense than "app name".

lgtm, overall. I will let @jonboulle resolve the image/name discussion with @cdaylward and @yifan-gu

@jonboulle I am ok with imageName, imageID, (do we need to change to imageLabels as well)?

Keeps the imageID, and change app to imageName.

@yifan-gu I don't think you just pulled my commit - the build is broken now :/

@jonboulle I did, and squashed the commits. For some reason we forgot this line :(
https://github.com/jonboulle/spec/commit/e343fa9f41ab0822001fe02bfde469872e8de6de#diff-dfef21140f3e6e95b72421f8887971d0R866

LGTM. Thanks.


Fix weather alarm

@chicco785 r?

@chicco785 I've managed to create a schema for Weather Alert reusing as much as possible of the Alert schema.
We have an issue with the build, as now Alert and Weather Alert schemas need to reference external ones. I don't know how is this configured in the validator, please could you help with that?
Thanks

@chicco785 now passing the tests. passing you the ball to do a review. thanks!

no news from @chicco785 ... should I auto-land or maybe someone can help? @fgalan


TRRT - TermRef processing doesn't seem to work (in eSSIF-Lab repo)

If you look at the 'Parties, Actors, Actions' section of the eSSIF-Lab framework, you see the following:

The source code of the first sentence of the Parties, Actors, Action section has TermRef [Parties, Actors and Actions pattern](party-actor-action@). Since there is only one MRGEntry with term: party-actor-action, this TermRef is correctly converted.
The source code of the first sentence of the Jurisdictions section has TermRef [Jurisdictions pattern](pattern:jurisdiction@), which is not resolved (if you hover over the link, you'll see that in the lower left corner of the browser screen). This also holds for the first sentence of the Guardianships section.

I believe this to be fixed now with TRRT v1.0.15

The first sentence in the guardianships section has a termref that doesn't get resolved properly. The termref specifically mentions pattern:guardianship, yet it resolves to concept:guardianship.

When I work locally, and generate the tools locally, (using TRRT vsn 1.0.15), I see that term-parts of term refs are not yet regularized. Here's some evidence

Fixed within TRRT v1.0.16. Within the HRGT and TRRT interpreters, predefined capturing groups weren't being regularized.


Feedback animation  response is very slow

Steps to reproduce

Select any field like popover

Expected results
response should be quick
Actual results
Animation feedback response  is slow
Platform
Web
Code sample
final fruits = {
'apple': 'Apple',
'banana': 'Banana',
'blueberry': 'Blueberry',
'grapes': 'Grapes',
'pineapple': 'Pineapple',
};
@override
Widget build(BuildContext context) {
return ConstrainedBox(
constraints: const BoxConstraints(minWidth: 180),
child: ShadSelect(
placeholder: const Text('Select a fruit'),
options: [
Padding(
padding: const EdgeInsets.fromLTRB(32, 6, 6, 6),
child: Text(
'Fruits',
style: theme.textTheme.muted.copyWith(
fontWeight: FontWeight.w600,
color: theme.colorScheme.popoverForeground,
),
textAlign: TextAlign.start,
),
),
...fruits.entries
.map((e) =>
ShadOption(value: e.key, child: Text(e.value)))
.toList(),
],
selectedOptionBuilder: (context, value) =>
Text(fruits[value]!),
),
);
}
Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
[Paste your output here]

@santoshgistto what do you mean with animation feedback?


go not working

$ go version
bash: /usr/local/go/bin/go: cannot execute binary file: Exec format error

How did you install Go? Perhaps you installed the wrong version.
Try file /usr/local/go/bin/go
And check that it matches your operation system.

On 7 Sep 2015, at 14:37, getpradeep9<EMAIL_ADDRESS>wrote:
$ go version
bash: /usr/local/go/bin/go: cannot execute binary file: Exec format error
—
Reply to this email directly or view it on GitHub.

Incomplete problem description. Please create a new issue with a proper description of the problem.

I had the same problem while executing golang program. I had installed 64-bit version of golang instead of 32-bit version. After installation of 32-bit OS version working fine.


com.orientechnologies.common.exception.OException: File flush was abnormally terminated

Hello,
We created a function to save a list of an Edge where we send a RID (FeedRID) and a list of users related with this. (5.000 users/request)
After some requests, this error below started. Then We should to restore a new database.
Do you know why it could be happened?
Error:

com.orientechnologies.common.exception.OException: File flush was abnormally terminated
java.util.concurrent.ExecutionException: java.io.IOException: Stream Closed
java.io.IOException: Stream Closed

function:

if(feedRID == null)
{
response.send(404, "Invalid feedRID", "text/plain", "Error: Invalid feedRID" );
}
else
{
var db = orient.getGraph();

try{
var feed = db.getVertex(feedRID);

var userArray = userList.split(",");

var index;
for (index = 0; index < userArray.length; ++index) {
var userId = userArray[index];

var d = new Date,
dformat = [d.getFullYear(),
d.getMonth()+1,
d.getDate()].join('-')+' '+
[d.getHours(),
d.getMinutes(),
d.getSeconds()].join(':');

var feedUser = db.addVertex("class:FeedUser");
feedUser.setProperty('FeedId', 0)
feedUser.setProperty('FeedUserId', 0)
feedUser.setProperty('LastActionDate', dformat)
feedUser.setProperty('UserId', userId)

db.addEdge(null, feed, feedUser, "FeedUserFeed");
}
  	return "Ok1";

}

catch ( err ){

	response.send(500, "Error on creating FeedUser", "text/plain", err.toString() );

}

}

Thank You,
Thiago

Hi Thiago
Can you send me your function?
Thanks
Valerio

of course:
function SaveFeedUser(feedRID, userList)
{
if(feedRID == null)
{
response.send(404, "Invalid feedRID", "text/plain", "Error: Invalid feedRID" );
}
else
{
var db = orient.getGraph();
try{
  	var feed = db.getVertex(feedRID);

  	var userArray = userList.split(",");

  	var index;
  	for (index = 0; index < userArray.length; ++index) {
  		var userId = userArray[index];

      var d = new Date,
          dformat = [d.getFullYear(),
           d.getMonth()+1,
           d.getDate()].join('-')+' '+
          [d.getHours(),
           d.getMinutes(),
           d.getSeconds()].join(':');

      var feedUser = db.addVertex("class:FeedUser");
      feedUser.setProperty('FeedId', 0)
      feedUser.setProperty('FeedUserId', 0)
      feedUser.setProperty('LastActionDate', dformat)
      feedUser.setProperty('UserId', userId)

    	db.addEdge(null, feed, feedUser, "FeedUserFeed");
  	}

  	return "Ok1";
}
catch ( err ){
	//db.rollback();
	response.send(500, "Error on creating FeedUser", "text/plain", err.toString() );
}

}
}

Hi Thiago could you send me your schema structure for replicate your problem?

Ok, I'm sending only classes used in the function, if you need the complete schema ask me please.
CREATE CLASS User EXTENDS V
CREATE PROPERTY User.UserId INTEGER
CREATE PROPERTY User.UserLoginId INTEGER
CREATE PROPERTY User.UserOrganizationId INTEGER
CREATE PROPERTY User.OrganizationId INTEGER
CREATE PROPERTY User.IsAdmin BOOLEAN
CREATE CLASS Feed EXTENDS V
CREATE PROPERTY Feed.FeedId INTEGER
CREATE PROPERTY Feed.Title STRING
CREATE PROPERTY Feed.Description STRING
CREATE PROPERTY Feed.FeedTypeId INTEGER
CREATE PROPERTY Feed.TaskTypeId INTEGER
CREATE PROPERTY Feed.CreateDate DATETIME
CREATE PROPERTY Feed.LessonCategoryId INTEGER
CREATE PROPERTY Feed.URL STRING
CREATE PROPERTY Feed.CardHtml STRING
CREATE PROPERTY Feed.UserId INTEGER
CREATE PROPERTY Feed.UserName STRING
CREATE PROPERTY Feed.ParentId INTEGER
CREATE CLASS FeedUser EXTENDS V
CREATE PROPERTY FeedUser.FeedUserId INTEGER
CREATE PROPERTY FeedUser.FeedId INTEGER
CREATE PROPERTY FeedUser.UserId INTEGER
CREATE PROPERTY FeedUser.LastActionDate DATETIME
CREATE CLASS FeedUserFeed EXTENDS E
CREATE CLASS UserFeedUser EXTENDS E
Thanks

Ow...I forgot this indexes:
CREATE INDEX User.UserId NOTUNIQUE_HASH_INDEX
CREATE INDEX Feed.FeedId NOTUNIQUE_HASH_INDEX
CREATE INDEX FeedUser.UserId NOTUNIQUE_HASH_INDEX

Hi Thiago, could you tell me which version are you using ?

We tested your function with your schema.
Try to do this changes:

adding db.commit () after you add the edge in the function;
change the type of property UserId in the FeedUser class from Integer to String;

Passing these parameters:
→ the RID of the feed (e.g. 13: 0)
→ the list of user (like this 12:0,12:1,12: 2)
the function generates the required edge.

Hi, this is bug which we chaining for a long time. I will provide distribution to find reason of it.

Please do not change code till we find root reason of issue, in any case, does not matter which code you use db should not be broken.

Could you do following:

Use this distrtibution https://www.dropbox.com/s/xs2w59sncv9ln4m/orientdb-community-2.1-SNAPSHOT-distribution.zip?dl=0
Set system property during server JVM start -Dfile.trackFileClose .
Repeat your test till you get java.io.IOException: Stream Closed
Send me<EMAIL_ADDRESS>server log.

I will look on it fix your issue.

@LucaSon90, I'm using v 2.1 rc5. Thank you.
@laa, I'll do it... Any news I tell you.
Thanks.

Hi,
Any updates on this ?

It is very critical issue for us, so once I have your data I will start to fix it.

Hi Andrey,
I sent you an email with a dump and logs, did you receive?
We got another error, but not this "stream closed".
We are trying, any update I tell you

Hi ,
Sorry but why do you think that OrientDB is blocked ?
I see that all threads waiting for schema reload, but this thread (reload of schema) is not frozen. It is working.
Also I see that you have huge amount of threads , it does not improve performance, it introduces huge context switching overhead.

I will provide you new distribution in few minutes, but still, are you sure that this is OK for you to use such amount of threads ?

Could you try this one https://www.dropbox.com/s/xs2w59sncv9ln4m/orientdb-community-2.1-SNAPSHOT-distribution.zip?dl=0  it contains fix to resolve you current bottleneck.
But still that is super important to reproduce stream closed exception.

OrientDB wasn't blocked, but wasn't answering the requests for almos an hour....
I can reduce pool size, today it's 40.
As we have 2 App servers and depending on the traffic we can have 20 Web servers.
Number of threads can go to 800.
Furthermore, the .Net driver always creates more connections when there isn't a free in the pool.

Ok, I'll start this version in some minutes.

I will be partially online so keep me posted.

This issue is only left for release so all development team is looking forward to fix this issue :-)

Man, We are trying to get this error again, but for while no way...
I'll keep it on till tomorrow... Sometimes we're getting only some http 500 when calling the function..

If each function creates 5k edges (5.000 users/request as you mentioned), could it be an out of memory problem? How much RAM do you have? How many concurrent call can be done to this function? Could you split the for() cycle to commit every X items, where X could be 500 for now? Example:
try{
    var feed = db.getVertex(feedRID);

    var userArray = userList.split(",");

    var index;
    for (index = 0; index < userArray.length; ++index) {
        var userId = userArray[index];

      var d = new Date,
          dformat = [d.getFullYear(),
           d.getMonth()+1,
           d.getDate()].join('-')+' '+
          [d.getHours(),
           d.getMinutes(),
           d.getSeconds()].join(':');

      var feedUser = db.addVertex("class:FeedUser");
      feedUser.setProperty('FeedId', 0)
      feedUser.setProperty('FeedUserId', 0)
      feedUser.setProperty('LastActionDate', dformat)
      feedUser.setProperty('UserId', userId)

        db.addEdge(null, feed, feedUser, "FeedUserFeed");

      if( index % 500 == 0 ) {
        db.commit();
        db.begin();
       }
    }
    db.commit();

    return "Ok1";
}
catch ( err ){
    //db.rollback();
    response.send(500, "Error on creating FeedUser", "text/plain", err.toString() );
}

Hi @lvca,
Yes we are having some memory problems either because of this function or some new queries. todays server have 15GB and we were getting this message since yesterday:
2015-07-23 07:13:05:506 INFO  -> Open server.sh (or server.bat on Windows) and c
hange the following variables: 1) MAXHEAP=-Xmx1434M 2) MAXDISKCACHE=11876 [OAbst
ractProfiler$MemoryChecker]
Today we got this:
2015-07-23 09:43:43:395 WARNI {db=ilang} Record (null) is null [OrientEdgeIterat
or]java.lang.OutOfMemoryError: GC overhead limit exceeded
Dumping heap to java_pid336.hprof ...
Heap dump file created [3224542164 bytes in 24.264 secs]
I talked to our developer that is developing this feature about some improvements on this, but as we were trying to get "java.io.IOException: Stream Closed" error, we hadn't changed yet.
Thanks of the tips.

@laa, sorry, but we couldn't reproduce the error yet...

OK, but seems I fixed it :-)


Allow course owner to upload video

I have successfully created the page for course owner to upload any videos. But the page was not achieved!

Now he is able to create new chapters.


no workflow instance

I deployed a DS Cluster minuts before, I ran a workflow manually and got a "Success" message, but I can't find anything in WorkFlow instace Tab,and the report showed no task had been submitted.
what's the problem could be
Which version of DolphinScheduler:
1.2.0
Additional context
The simplest workflow I defined

set the task online and run it, I got "success" msg

but there is nothing in workflow instance Tab

and I haved checked the log,I got these Errors,dose it mater

**Requirement or improvement

Please describe about your requirements or improvement suggestions.

the repoort shows that no task has been submitted

Please check the master service and worker service logs

1 use command jps ，check if there are 5 processes
10517 MasterServer
10550 WorkerServer
10583 LoggerServer
10616 AlertServer
10649 ApiApplicationServer

2 if exist , please check Master and  Worker log，does it contain the following logs?
[WARN] 2020-04-19 22:45:53.462 org.apache.dolphinscheduler.common.utils.OSUtils:[295] - load or availablePhysicalMemorySize(G) is too high, it's availablePhysicalMemorySize(G):0.98,loadAvg:0.05
3 if contains the log, that is server not enough memory or CPU,you can modify configuration
the file dir  bin/dolphinscheduler-daemon.sh
modify param  -Xmx and -Xms
export DOLPHINSCHEDULER_OPTS="-server -Xmx300m -Xms100m -Xss512k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:LargePageSizeInBytes=128m -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70"

thank u guys,
the problem is solved after I reconfig DOLPHINSCHEDULER_OPTS as:
export DOLPHINSCHEDULER_OPTS="-server -Xmx300m -Xms100m -Xss512k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:LargePageSizeInBytes=128m -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70"

thank u guys,
the problem is solved after I reconfig DOLPHINSCHEDULER_OPTS as:
export DOLPHINSCHEDULER_OPTS="-server -Xmx300m -Xms100m -Xss512k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:LargePageSizeInBytes=128m -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70"

the param  -server -Xmx300m -Xms100m is my local test environment configuration, you need to allocate memory according to the actual situation


Migrate setup-java action to use Temurin

AdoptOpenJDK has moved to the Eclipse Foundation and now distributes binaries under the new name "Eclipse Temurin"

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Unauthorized(401) when using tweets/search/recent

Prerequisites

[x] Put an X between the brackets on this line if you have done all of the
following:

Checked that your issue isn't already filed: open issues
Checked the bug report guidelines

Description
I'm trying to get the recent tweets from an user excluding the retweets.
The documentation says that I can use -is:retweet on my query and I tried this:
const client = new Twitter({
  consumer_key: process.env.TWITTER_API_KEY,
  consumer_secret: process.env.TWITTER_API_SECRET,
  access_token_key: process.env.TWITTER_ACCESS_TOKEN,
  access_token_secret: process.env.TWITTER_ACCESS_TOKEN_SECRET,
});

const getLatestTweets = async (limit = 20) => {
  const { data } = await client.get('tweets/search/recent', {
    query: 'from:levxyca -is:retweet',
    'tweet.fields': 'created_at,public_metrics,entities',
    max_results: limit,
  });

  if (!data) { /* Not important for this issue. :D */ }

  return data;
};

getLatestTweets();

The point is on query: 'from:levxyca -is:retweet'. If a change it to query: 'from:levxyca', I can get the user tweets but I need to remove the retweets manually. When I add the -is:retweet I get the error specified below.
Do I need to do any dealings with my query?
Actual behavior:
I'm getting a Unauthorized error:
:\Users\Me\Desktop\Repositories\pandadomalbot\node_modules\twitter-v2\build\TwitterError.js:23
        return new module.exports(json.title, json.status, json.detail);
               ^

TwitterError: Unauthorized
    at Function.module.exports.fromJson (C:\Users\Me\Desktop\Repositories\pandadomalbot\node_modules\twitter-v2\build\TwitterError.js:23:16)
    at Twitter.get (C:\Users\Me\Desktop\Repositories\pandadomalbot\node_modules\twitter-v2\build\twitter.js:43:49)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async getLatestTweets (C:\Users\Me\Desktop\Repositories\pandadomalbot\utils\twitter.js:35:20)
    at async getTodaysLiveAnnouncement (C:\Users\Me\Desktop\Repositories\pandadomalbot\utils\twitter.js:70:18)
    at async Object.exports.default (C:\Users\Me\Desktop\Repositories\pandadomalbot\commands\rt.js:9:19) {
  code: 401,
  details: 'Unauthorized'
}

Versions
1.1.0

Thanks for reporting this. I've noticed that the 401 gets returned if the query is anything but a simple word. For example changing query to obama biden causes it too.

I've noticed that it will work if you don't configure your client with access_token_key and access_token_secret? This is what the twitter-v2 unit test is doing which passes with the complex query 'url:"https://medium.com" -is:retweet lang:en'.
I wonder if the v2 API rules around what can be done in queries are coming into play here? I wouldn't be completely surprised if it's a bug on Twitter's side too.
https://developer.twitter.com/en/docs/twitter-api/tweets/search/integrate/build-a-query#availability

Testing the equivalent query in Python leads me to the conclusion that the problem resides here in twitter-v2 and not on the Twitter side.

I've added a unit test, which is currently failing:
https://github.com/edsu/twitter-v2/blob/master/test/e2e/search.js#L72-L105

I might be wrong, but it appears that the space in the query parameter gets converted to a + when calling URL.toString() which is correct. But somewhere in the underlying call to oauth-1.0a/OAuth.authorize here any + in the URL are being converted to %20 prior to signing?  This is causing the request to be invalid.

More notes on this (sorry if they aren't helping). It looks like there's a mismatch in how oauth-1.0a and twitter-v2 are encoding spaces ' '.
The problem is that URL.toString() encodes a space ' ' in a query paramter as + whereas encodeURIComponent() encodes a space ' ' as %20.
> encodeURIComponent(' ')
'%20'
> u = new URL('http://example.com')
URL {
  href: 'http://example.com/',
  origin: 'http://example.com',
  protocol: 'http:',
  username: '',
  password: '',
  host: 'example.com',
  hostname: 'example.com',
  port: '',
  pathname: '/',
  search: '',
  searchParams: URLSearchParams {},
  hash: ''
}
> u.searchParams.set('foo', 'bar baz')
undefined
> u.toString()
'http://example.com/?foo=bar+baz'

I have submitted a PR with a fix #93. I really depend on this library in one of my projects and have adjusted my package.json to point to edsu/twitter-v2 until some kind of fix is available here.

@edsu In one of the projects that I contribute, I also switched to use your fork too. Thanks!


improve documentation for supported elements

motivation
the bullet items of supported elements is quite compact and not easy to digest for beginners.
so, the main idea is to improve documentation for beginners
acceptance criteria

[ ] pictures/symbols for each elements are shown
[ ] short element description is available
[ ] relevant caveats (compared to other engines) are described

Hi @nitram509 , what are the current supported elements?

@rhzs just have a look here
https://nitram509.github.io/lib-bpmn-engine/supported-elements/
source:
https://github.com/nitram509/lib-bpmn-engine/blob/main/docs/supported-elements.md
I can support with the last acceptance criteria ;)


fix: add skeleton for kafka details page

What this PR does / why we need it:
add skeleton for kafka details page
Which issue(s) this PR fixes

Replace this comment with references to related issues.
e.g. fixes #<issue number>

Verification steps

Attach a screenshot of the output.
guidance on how to go the particular story.
e.g. point to the story with a text breadcrumb, something like "components > <component name> > <story name>”

Is there any work left / what's next :

Replace this comment with pending items that need to be covered in this PR or some pending items you are planning to do in the future PR.

Special notes for your reviewer:

:tada: This PR is included in version 1.36.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


BLE pycom is not visible on a bLEscanner

when I ran SampleServerEncryption to use my ESP32 as a server nothing is displayed if I do a scan with my tel using a BLEscanner And i can stop monitor with ctrl + c .
I use eclipse c++ on mac os x  and terminal to display monitor.
Need Help

Could you paste logs from esp32?

le: 0x28
D (915) FreeRTOS: Semaphore taking: name: CreateEvt (0x3ffd23b0), owner: <N/A> for executeCreate
D (923) FreeRTOS: Semaphore taken:  name: CreateEvt (0x3ffd23b0), owner: executeCreate
E (931) BT: osi_mem_dbg_record invalid !!

D (935) BLEDevice: gattServerEventHandler [esp_gatt_if: 3] ... ESP_GATTS_ADD_CHAR_EVT
D (942) BLEUtils: GATT ServerEvent: ESP_GATTS_ADD_CHAR_EVT
D (948) BLEUtils: [status: ESP_GATT_OK, attr_handle: 42 0x2a, service_handle: 40 0x28, char_uuid: 0eea3ca1-782b-47b9-bbcd-dbbddf10a38e]
D (959) BLEServer: >> handleGATTServerEvent: ESP_GATTS_ADD_CHAR_EVT
D (965) BLECharacteristic: >> setHandle: handle=0x2a, characteristic uuid=0eea3ca1-782b-47b9-bbcd-dbbddf10a38e
D (975) BLECharacteristic: << setHandle
V (979) FreeRTOS: Semaphore giving: name: CreateEvt (0x3ffd23b0), owner: executeCreate
D (986) BLEServer: << handleGATTServerEvent
V (990) FreeRTOS: >> wait: Semaphore waiting: name: CreateEvt (0x3ffd23b0), owner: <N/A> for executeCreate
V (1000) FreeRTOS: << wait: Semaphore released: name: CreateEvt (0x3ffd23b0), owner: executeCreate
D (1008) BLEDescriptor: >> executeCreate(): UUID: 00002902-0000-1000-8000-00805f9b34fb, handle: 0xffff
D (1017) FreeRTOS: Semaphore taking: name: CreateEvt (0x3ffd24fc), owner: <N/A> for executeCreate
D (1026) FreeRTOS: Semaphore taken:  name: CreateEvt (0x3ffd24fc), owner: executeCreate
D (1034) BLEDevice: gattServerEventHandler [esp_gatt_if: 3] ... ESP_GATTS_ADD_CHAR_DESCR_EVT
D (1042) BLEUtils: GATT ServerEvent: ESP_GATTS_ADD_CHAR_DESCR_EVT
D (1048) BLEUtils: [status: ESP_GATT_OK, attr_handle: 43 0x2b, service_handle: 40 0x28, char_uuid: 00002902-0000-1000-8000-00805f9b34fb]
D (1060) BLEServer: >> handleGATTServerEvent: ESP_GATTS_ADD_CHAR_DESCR_EVT
D (1066) BLEDescriptor: >> setHandle(0x2b): Setting descriptor handle to be 0x2b
D (1073) BLEDescriptor: << setHandle()
V (1077) FreeRTOS: Semaphore giving: name: CreateEvt (0x3ffd24fc), owner: executeCreate
D (1085) BLEServer: << handleGATTServerEvent
V (1089) FreeRTOS: >> wait: Semaphore waiting: name: CreateEvt (0x3ffd24fc), owner: <N/A> for executeCreate
V (1098) FreeRTOS: << wait: Semaphore released: name: CreateEvt (0x3ffd24fc), owner: executeCreate
D (1107) BLEDescriptor: << executeCreate
D (1110) BLECharacteristic: << executeCreate
D (1114) FreeRTOS: Semaphore taking: name: StartEvt (0x3ffd1ff8), owner: <N/A> for start
D (1122) FreeRTOS: Semaphore taken:  name: StartEvt (0x3ffd1ff8), owner: start
D (1130) BLEDevice: gattServerEventHandler [esp_gatt_if: 3] ... ESP_GATTS_START_EVT
D (1136) BLEUtils: GATT ServerEvent: ESP_GATTS_START_EVT
D (1141) BLEUtils: [status: ESP_GATT_OK, service_handle: 0x28]
D (1147) BLEServer: >> handleGATTServerEvent: ESP_GATTS_START_EVT
V (1153) FreeRTOS: Semaphore giving: name: StartEvt (0x3ffd1ff8), owner: start
D (1160) BLEServer: << handleGATTServerEvent
V (1164) FreeRTOS: >> wait: Semaphore waiting: name: StartEvt (0x3ffd1ff8), owner: <N/A> for start
V (1172) FreeRTOS: << wait: Semaphore released: name: StartEvt (0x3ffd1ff8), owner: start
D (1180) BLEService: << start()
D (1183) SampleServer: Advertising started!
D (1187) BLEAdvertising: >> start: customAdvData: 0, customScanResponseData: 0
D (1194) BLEAdvertising: - advertising service: 9845f7dc-4468-491a-9d8a-d3aba0eb8287
D (1204) BLEUtils: Received a GAP event: ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT
D (1209) BLEUtils: [status: 0]
D (1211) BLEServer: BLEServer ... handling GAP event!
D (1217) BLEUtils: Received a GAP event: ESP_GAP_BLE_SCAN_RSP_DATA_SET_COMPLETE_EVT
D (1223) BLEUtils: [status: 0]
D (1226) BLEServer: BLEServer ... handling GAP event!
D (1233) BLEUtils: Received a GAP event: ESP_GAP_BLE_ADV_START_COMPLETE_EVT
D (1238) BLEUtils: [status: 0]
D (1240) BLEServer: BLEServer ... handling GAP event!
D (1245) BLEAdvertising: << start
D (1248) SampleServer: Advertising started!

From logs everything seems to be fine. Advertising starts here. Can you try with different application on you phone or maybe with laptop or something?
D (1194) BLEAdvertising: - advertising service: 9845f7dc-4468-491a-9d8a-d3aba0eb8287
D (1204) BLEUtils: Received a GAP event: ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT
D (1209) BLEUtils: [status: 0]
D (1211) BLEServer: BLEServer ... handling GAP event!
D (1217) BLEUtils: Received a GAP event: ESP_GAP_BLE_SCAN_RSP_DATA_SET_COMPLETE_EVT
D (1223) BLEUtils: [status: 0]
D (1226) BLEServer: BLEServer ... handling GAP event!
D (1233) BLEUtils: Received a GAP event: ESP_GAP_BLE_ADV_START_COMPLETE_EVT
D (1238) BLEUtils: [status: 0]
D (1240) BLEServer: BLEServer ... handling GAP event!
D (1245) BLEAdvertising: << start
D (1248) SampleServer: Advertising started!

Yeah i see all is correct.
I try with the application of Nordit on my 2 phones (apple and android) nothing is found.
And my problem is also the monitor does not close if I want to close it with cmd + c or other as if it is planted that I disconnect my esp32 if I want to close.

To see all monitor option use:
cmd + T and cmd + H (it works on linux)
About SampleServer_Encrypted, i cant help you with that. Just checked and everything seems works fine. Compiled, installed and found with laptop and android phone (nRF connect).

But I want to know if there is an application other than nrf that we can have on pc ?
Cause if i use micropython create an server with BLE its found with nrf nordic and i can see caracterist and others.

Like i said, i can see SampleServer_Encrypted and any other server examples with my windows 10 laptop. I dont have mac OS so i cant confirm or deny its working, but its a standard so it should have works.

of course i will try to see on my windows 10 because I had problems with my esp32 drivers.
But I will try to see for the configuration for the drivers under esp32 thanks again for everything.
Best Regards

No, i say that i have problem to use MSYS2 MINGW32 under windows I could not browse to finish the installation that's why I abandoned windows.


Populate submitted values in site details

Issue #1038
Status and custom spotter api token values are not displayed in site details.
Changes

Add status initial value
Change api to return the spotter api token masked (only 4 last digits) if it has custom token. Unfortunately we cannot add a new computed property in the Site entity, because the spotterApiToken is not selected and therefore accessible by default, so we select all site properties and whitelist the properties we want to return.
On UI, show the masked token. User can provide a new one by clicking on the input field - then the old token is erased and the user can start typing the new token. If they leave the input without typing anything, the old token is filled back in. If the user wants to completely remove the custom token they uncheck the "Add custom api token" checkbox.

Note the logged spotterApiToken, when null the token is deleted, when undefined it's not changed and when it has value, the value is persisted.
Screencast from 23-10-2024 05:16:05 ΜΜ.webm

Hi @K-Markopoulos,
Thank you for working on this. When testing it out, I added a spotter (not owned by Aqualink) and added its API token. I updated the page and went back to the dashboard. You can see the screenshot below. The Status "deployed" works fine, but I can not see the "Add a custom API token" box being checked or any value in the token textbox.
Regarding the second point in your changes, I didn’t fully understand it. If it's possible to show the last values that were entered for the API token, that's great. If not, that's fine and no need to do too much work on it. This view is locked to the SuperAdmin and the Site Manager for the specific site, so there's no need to mask the token or anything fancy (unless it has already been done). The "EDIT SITE DETAILS" view is only for the site manager of the specific site and the SuperAdmins to keep track of and troubleshoot what data has been entered into the database.

Hi @Caesarh97,
If I'm not wrong, the generated preview link (https://aqualink-app-1043.surge.sh/) includes only the frontend changes of the PR. The backend changes have to be manually deployed to be available/testable and the spotter api token fix has some backend changes. @ericboucher can you help us with the backend deployment in order to test this PR?
The 2nd point is describing some technical things about the implementation. Basically, with the current implementation, once the token is persisted, it won't be fully visible in the UI. The user will be able to see only the last 4 characters of it. If they set "123456789", they will see "****6789".

Hi @Caesarh97, If I'm not wrong, the generated preview link (https://aqualink-app-1043.surge.sh/) includes only the frontend changes of the PR. The backend changes have to be manually deployed to be available/testable and the spotter api token fix has some backend changes. @ericboucher can you help us with the backend deployment in order to test this PR?
The 2nd point is describing some technical things about the implementation. Basically, with the current implementation, once the token is persisted, it won't be fully visible in the UI. The user will be able to see only the last 4 characters of it. If they set "123456789", they will see "****6789".


unlikely button spotted

I am working on this issue.

@Pratik10100 I'm not sure what the issue you are reporting is. Please include a detailed description of the issue, including at least the following: written description describing the issue, written description of what should be improved and why, detailed steps to reproduce, which browser and platform the issue was found on. Also when adding images, please make sure that the descriptions are enough to be understood without the image being viewed. It may be the case that someone reading this issue cannot perceive the image. For me, I am able to perceive the image, but am still not sure what I'm supposed to be looking at.


transaction rewrite, increase pg connection timeout

https://github.com/omisego/elixir-omg/issues/1378#issuecomment-632642904
Overview
Rewritten the transaction into ectomulti.
Changes
Describe your changes and implementation choices. More details make PRs easier to review.

Moved some tests around
Rewrote the transaction as Ecto Multi

Testing
mix test

Coverage decreased (-11.2%) to 66.994% when pulling c4d87b2e3e09097fe13b67ca87552191df982bc9 on inomurko/1378-applyblock into dd5f33de10b7f34d334f8047954b7b1db7f9a13d on master.

@InoMurko Where you able to try this out with a 64k transactions block? Where would be the best place for us to introduce a test like this to ensure we don't break it in the future?

My testing locally was with 50k transactions. Let me just reiterate that this PR doesn't introduce any performance increases.
https://github.com/omisego/elixir-omg/issues/1378#issuecomment-632642904

@InoMurko understood, but you did introduce a bigger timeout, which appears to solve the issue with 50k transactions, right? That's good, but not the maximum we can have in a block so we probably need to test with a bit more 🙇 . And I still would like to see those changes documented in a test ran somewhere in the pipeline. cc @kevsul

Explaining the test in SQL terms:
At first, we insert a block https://github.com/omisego/elixir-omg/pull/1525/files#diff-a45c5891aed43f4e5b24439fb6e4ce70R169
INSERT INTO "blocks" ("blknum","eth_height","hash","timestamp","inserted_at","updated_at") VALUES ($1,$2,$3,$4,$5,$6) [1000, 1, "0x1000",<PHONE_NUMBER>, ~U[2020-05-25 16:18:45.656007Z], ~U[2020-05-25 16:18:45.656007Z]]⋅
2020-05-25 18:18:46.660 [debug] module=Ecto.Adapters.SQL function=log/4 ⋅QUERY OK db=468.3ms

Transactions and outputs that "came out of that block". These are being chunked into equal parts. Where a single part has at most 0xFFFF items:
https://github.com/omisego/elixir-omg/pull/1525/files#diff-a45c5891aed43f4e5b24439fb6e4ce70R170-R171
Transactions:
INSERT INTO "transactions" ("blknum","inserted_at","metadata","txbytes","txhash","txindex","txtype","updated_at") VALUES ($1,$2,$3,$4,$5,$6,$7,$8),($9,$10,$11,$12,$13,$14,$15,$16),($17,$18,$19,$20,$21,$22,$23,$24),($25,$26,$27,$28,$29,$30,$31,$32),($33,$34,$35,$36,$37,$38,$39,$40),($41,$42,$43,$44,$45,$46,$47,$48),($49,$50,$51,$52,$53,$54,$55,$56),($57,$58,$59,$60,$61,$62,$63,$64),($65,$66,$67,$68,$69,$70,$71,$72),($73,$74,$75,$76,$77,$78,$79,$80),($81,$82,$83,$84,$85,$86,$87,$88),($89,$90,$91,$92,$93,$94,$95,$96),($97,$98,$99,$100,$101,$102,$103,$104),($105,$106,$107,$108,$109,$110,$111,$112),($113,$114,$115,$116,$117,$118,$119,$120),($121,$122,$123,$124,$125,$126,$127,$128),($129,$130,$131,$132,$133,$134,$135,$136),($137,$138,$139,$140,$141,$142,$143,$144),($145,$146,$147,$148,$149,$150,$151,$152),($153,$154,$155,$156,$157,$158,$159,$160),($161,$162,$163,$164,$165,$166,$167,$168),($169,$170,$171,$172,$173,$174,$175,$176),($177,$178,$179,$180,$181,$182,$183,$184),($185,$186,$187,$188,$189,$190,$191,$192),($193,$194,$195,$196,$197,$198,$199,$200),($201,$202,$203,$204,$205,$206,$207,$208),($209,$210,$211,$212,$213,$214,$215,$216),($217,$218,$219,$220,$221,$222,$223,$224),($225,$226,$227,$228,$229,$230,$231,$232),($233,$234,$235,$236,$237,$238,$239,$240),($241,$242,$243,$244,$245,$246,$247,$248),($249,$250,$251,$252,$253,$254,$255,$256),($257,$258,$259,$260,$261,$262,$263,$264),($265,$266,$267,$268,$269,$270,$271,$272),($273,$274,$275,$276,$277,$278,$279,$280),($281,$282,$283,$284,$285,$286,$287,$288),($289,$290,$291,$292,$293,$294,$295,$296),($297,$298,$299,$300,$301,$302,$303,$304),($305,$306,$307,$308,$309,$310,$311,$312),($313,$314,$315,$316,$317,$318,$319,$320),($321,$322,$323,$324,$325,$326,$327,$328),($329,$330,$331,$332,$333,$334,$335,$336),($337,$338,$339,$340,$341,$342,$343,$344),($345,$346,$347,$348,$349,$350,$351,$352),($353,$354,$355,$356,$357,$358,$359,$360),($361,$362,$363,$364,$365,$366,$367,$368),($369,$370,$371,$372,$373,$374,$375,$376),($377,$378,$379,$380,$381,$382,$383,$384),($385,$386,$387,$388,$389,$390,$391,$392),($393,$394,$395,$396,$397,$398,$399,$400),($401,$402,$403,$404,$405,$406,$407,$408),($409,$410,$411,$412,$413,$414,$415,$416),($417,$418,$419,$420,$421,$422,$423,$424),($425,$426,$427,$428,$429,$430,$431,$432),($433,$434,$435,$436,$437,$438,$439,$440),($441,$442,$443,$444,$445,$446,$447,$448),($449,$450,$451,$452,$453,$454,$455,$456),($457,$458,$459,$460,$461,$462,$463,$464),($465,$466,$467,$468,$469,$470,$471,$472),($473,$474,$475,$476,$477,$478,$479,$480),($481,$482,$483,$484,$485,$486,$487,$488),($489,$490,$491,$492,$493,$494,$495,$496),($497,$498,$499,$500,$501,$502,$503,$504),($505,$506,$507,$508,$509,$510,$511,$512),($513,$514,$515,$516,$517,$518,$519,$520),($521,$522,$523,$524,$525,$526,$527,$528),($529,$530,$531,$532,$533,$534,$535,$536),($537,$538,$539,$540,$541,$542,$543,$544),($545,$546,$547,$548,$549,$550,$551,$552),($553,$554,$555,$556,$557,$558,$559,$560),($561,$562,$563,$564,$565,$566,$567,$568),($569,$570,$571,$572,$573,$574,$575,$576),($577,$578,$579,$580,$581,$582,$583,$584),($585,$586,$587,$588,$589,$590,$591,$592),($593,$594,$595,$596,$597,$598,$599,$600),($601,$602,$603,$604,$605,$606,$607,$608),($609,$610,$611,$612,$613,$614,$615,$616),($617,$618,$619,$620,$621,$622,$623,$624),($625,$626,$627,$628,$629,$630,$631,$632),($633,$634,$635,$636,$637,$638,$639,$640),($641,$642,$643,$644,$645,$646,$647,$648),($649,$650,$651,$652,$653,$654,$655,$656),($657,$658,$659,$660,$661,$662,$663,$664),($665,$666,$667,$668,$669,$670,$671,$672),($673,$674,$675,$676,$677,$678,$679,$680),($681,$682,$683,$684,$685,$686,$687,$688),($689,$690,$691,$692,$693,$694,$695,$696),($697,$698,$699,$700,$701,$702,$703,$704),($705,$706,$707,$708,$709,$710,$711,$712),($713,$714,$715,$716,$717,$718,$719,$720),($721,$722,$723,$724,$725,$726,$727,$728),($729,$730,$731,$732,$733,$734,$735,$736),($737,$738,$739,$740,$741,$742,$743,$744),($745,$746,$747,$748,$749,$750,$751,$752),($753,$754,$755,$756,$757,$758,$759,$760),($761,$762,$763,$764,$765,$766,$767,$768),($769,$770,$771,$772,$773,$774,$775,$776),($777,$778,$779,$780,$781,$782,$783,$784),($785,$786,$787,$788,$789,$790,$791,$792),($793,$794,$795,$796,$797,$798,$799,$800),($801,$802,$803,$804,$805,$806,$807,$808),($809,$810,$811,$812,$813,$814,$815,$816),($817,$818,$819,$820,$821,$822,$823,$824),($825,$826,$827,$828,$829,$830,$831,$832),($833,$834,$835,$836,$837,$838,$839,$840),($841,$842,$843,$844,$845,$846,$847,$848),($849,$850,$851,$852,$853,$854,$855,$856),($857,$858,$859,$860,$861,$862,$863,$864),($865,$866,$867,$868,$869,$870,$871,$872),($873,$874,$875,$876,$877,$878,$879,$880),($881,$882,$883,$884,$885,$886,$887,$888),($889,$890,$891,$892,$893,$894,$895,$896),($897,$898,$899,$900,$901,$902,$903,$904),($905,$906,$907,$908,$909,$910,$911,$912),($913,$914,$915,$916,$917,$918,$919,$920),($921,$922,$923,$924,$925,$926,$927,$928),($929,$930,$931,$932,$933,$934,$935,$936),($937,$938,$939,$940,$941,$942,$943,$944),($945,$946,$947,$948,$949,$950,$951,$952),($953,$954,$955,$956,$957,$958,$959,$960),($961,$962,$963,$964,$965,$966,$967,$968),($969,$970,$971,$972,$973,$974,$975,$976),($977,$978,$979,$980,$981,$982,$983,$984),($985,$986,$987,$988,$989,$990,$991,$992),($993,$994,$995,$996,$997,$998,$999,$1000),($1001,$1002,$1003,$1004,$1005,$1006,$1007,$1008),($1009,$1010,$1011,$1012,$1013,$1014,$1015,$1016),($1017,$1018,$1019,$1020,$1021,$1022,$1023,$1024),($1025,$1026,$1027,$1028,$1029,$1030,$1031,$1032),($1033,$1034,$1035,$1036,$1037,$1038,$1039,$1040),($1041,$1042,$1043,$1044,$1045,$1046,$1047,$1048),($1049,$1050,$1051,$1052,$1053,$1054,$1055,$1056),($1057,$1058,$1059,$1060,$1061,$1062,$1063,$1064),($1065,$1066,$1067,$1068,$1069,$1070,$1071,$1072),($1073,$1074,$1075,$1076,$1077,$1078,$1079,$1080),($1081,$1082,$1083,$1084,$1085,$1086,$1087,$1088),($1089,$1090,$1091,$1092,$1093,$1094,$1095,$1096),($1097,$1098,$1099,$1100,$1101,$1102,$1103,$1104),($1105,$1106,$1107,$1108,$1109,$1110,$1111,$1112),($1113,$1114,$1115,$1116,$1117,$1118,$1119,$1120),($1121,$1122,$1123,$1124,$1125,$1126,$1127,$1128),($1129,$1130,$1131,$1132,$1133,$1134,$1135,$1136),($1137,$1138,$1139,$1140,$1141,$1142,$1143,$1144),($1145,$1146,$1147,$1148,$1149,$1150,$1151,$1152),($1153,$1154,$1155,$1156,$1157,$1158,$1159,$1160),($1161,$1162,$1163,$1164,$1165,$1166,$1167,$1168),($1169,$1170,$1171,$1172,$1173,$1174,$1175,$1176),($1177,$1178,$1179,$1180,$1181,$1182,$1183,$1184),($1185,$1186,$1187,$1188,$1189,$1190,$1191,$1192),($1193,$1194,$1195,$1196,$1197,$1198,$1199,$1200),($1201,$1202,$1203,$1204,$1205,$1206,$1207,$1208),($1209,$1210,$1211,$1212,$1213,$1214,$1215,$1216),($1217,$1218,$1219,$1220,$1221,$1222,$1223,$1224),($1225,$1226,$1227,$1228,$1229,$1230,$1231,$1232),($1233,$1234,$1235,$1236,$1237,$1238,$1239,$1240),($1241,$1242,$1243,$1244,$1245,$1246,$1247,$1248),($1249,$1250,$1251,$1252,$1253,$1254,$1255,$1256),($1257,$1258,$1259,$1260,$1261,$1262,$1263,$1264),($1265,$1266,$1267,$1268,$1269,$1270,$1271,$1272),($1273,$1274,$1275,$1276,$1277,$1278,$1279,$1280),($1281,$1282,$1283,$1284,$1285,$1286,$1287,$1288),($1289,$1290,$1291,$1292,$1293,$1294,$1295,$1296),($1297,$1298,$1299,$1300,$1301,$1302,$1303,$1304),($1305,$1306,$1307,$1308,$1309,$1310,$1311,$1312),($1313,$1314,$1315,$1316,$1317,$1318,$1319,$1320),($1321,$1322,$1323,$1324,$1325,$1326,$1327,$1328),($1329,$1330,$1331,$1332,$1333,$1334,$1335,$1336),($1337,$1338,$1339,$1340,$1341,$1342,$1343,$1344),($1345,$1346,$1347,$1348,$1349,$1350,$1351,$1352),($1353,$1354,$1355,$1356,$1357,$1358,$1359,$1360),($1361,$1362,$1363,$1364,$1365,$1366,$1367,$1368),($1369,$1370,$1371,$1372,$1373,$1374,$1375,$1376),($1377,$1378,$1379,$1380,$1381,$1382,$1383,$1384),($1385,$1386,$1387,$1388,$1389,$1390,$1391,$1392),($1393,$1394,$1395,$1396,$1397,$1398,$1399,$1400),($1401,$1402,$1403,$1404,$1405,$1406,$1407,$1408),($1409,$1410,$1411,$1412,$1413,$1414,$1415,$1416),($1417,$1418,$1419,$1420,$1421,$1422,$1423,$1424),($1425,$1426,$1427,$1428,$1429,$1430,$1431,$1432),($1433,$1434,$1435,$1436,$1437,$1438,$1439,$1440),($1441,$1442,$1443,$1444,$1445,$1446,$1447,$1448),($1449,$1450 (truncated)⋅
same stuff, but for txoutputs
INSERT INTO "txoutputs" ("amount","blknum","creating_txhash","currency","inserted_at","oindex","otype","owner","txindex","updated_at") VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10),($11,$12,$13,$14,$15,$16,$17,$18,$19,$20),($21,$22,$23,$24,$25,$26,$27,$28,$29,$30),($31,$32,$33,$34,$35,$36,$37,$38,$39,$40),($41,$42,$43,$44,$45,$46,$47,$48,$49,$50),($51,$52,$53,$54,$55,$56,$57,$58,$59,$60),($61,$62,$63,$64,$65,$66,$67,$68,$69,$70),($71,$72,$73,$74,$75,$76,$77,$78,$79,$80),($81,$82,$83,$84,$85,$86,$87,$88,$89,$90),($91,$92,$93,$94,$95,$96,$97,$98,$99,$100),($101,$102,$103,$104,$105,$106,$107,$108,$109,$110),($111,$112,$113,$114,$115,$116,$117,$118,$119,$120),($121,$122,$123,$124,$125,$126,$127,$128,$129,$130),($131,$132,$133,$134,$135,$136,$137,$138,$139,$140),($141,$142,$143,$144,$145,$146,$147,$148,$149,$150),($151,$152,$153,$154,$155,$156,$157,$158,$159,$160),($161,$162,$163,$164,$165,$166,$167,$168,$169,$170),($171,$172,$173,$174,$175,$176,$177,$178,$179,$180),($181,$182,$183,$184,$185,$186,$187,$188,$189,$190),($191,$192,$193,$194,$195,$196,$197,$198,$199,$200),($201,$202,$203,$204,$205,$206,$207,$208,$209,$210),($211,$212,$213,$214,$215,$216,$217,$218,$219,$220),($221,$222,$223,$224,$225,$226,$227,$228,$229,$230),($231,$232,$233,$234,$235,$236,$237,$238,$239,$240),($241,$242,$243,$244,$245,$246,$247,$248,$249,$250),($251,$252,$253,$254,$255,$256,$257,$258,$259,$260),($261,$262,$263,$264,$265,$266,$267,$268,$269,$270),($271,$272,$273,$274,$275,$276,$277,$278,$279,$280),($281,$282,$283,$284,$285,$286,$287,$288,$289,$290),($291,$292,$293,$294,$295,$296,$297,$298,$299,$300),($301,$302,$303,$304,$305,$306,$307,$308,$309,$310),($311,$312,$313,$314,$315,$316,$317,$318,$319,$320),($321,$322,$323,$324,$325,$326,$327,$328,$329,$330),($331,$332,$333,$334,$335,$336,$337,$338,$339,$340),($341,$342,$343,$344,$345,$346,$347,$348,$349,$350),($351,$352,$353,$354,$355,$356,$357,$358,$359,$360),($361,$362,$363,$364,$365,$366,$367,$368,$369,$370),($371,$372,$373,$374,$375,$376,$377,$378,$379,$380),($381,$382,$383,$384,$385,$386,$387,$388,$389,$390),($391,$392,$393,$394,$395,$396,$397,$398,$399,$400),($401,$402,$403,$404,$405,$406,$407,$408,$409,$410),($411,$412,$413,$414,$415,$416,$417,$418,$419,$420),($421,$422,$423,$424,$425,$426,$427,$428,$429,$430),($431,$432,$433,$434,$435,$436,$437,$438,$439,$440),($441,$442,$443,$444,$445,$446,$447,$448,$449,$450),($451,$452,$453,$454,$455,$456,$457,$458,$459,$460),($461,$462,$463,$464,$465,$466,$467,$468,$469,$470),($471,$472,$473,$474,$475,$476,$477,$478,$479,$480),($481,$482,$483,$484,$485,$486,$487,$488,$489,$490),($491,$492,$493,$494,$495,$496,$497,$498,$499,$500),($501,$502,$503,$504,$505,$506,$507,$508,$509,$510),($511,$512,$513,$514,$515,$516,$517,$518,$519,$520),($521,$522,$523,$524,$525,$526,$527,$528,$529,$530),($531,$532,$533,$534,$535,$536,$537,$538,$539,$540),($541,$542,$543,$544,$545,$546,$547,$548,$549,$550),($551,$552,$553,$554,$555,$556,$557,$558,$559,$560),($561,$562,$563,$564,$565,$566,$567,$568,$569,$570),($571,$572,$573,$574,$575,$576,$577,$578,$579,$580),($581,$582,$583,$584,$585,$586,$587,$588,$589,$590),($591,$592,$593,$594,$595,$596,$597,$598,$599,$600),($601,$602,$603,$604,$605,$606,$607,$608,$609,$610),($611,$612,$613,$614,$615,$616,$617,$618,$619,$620),($621,$622,$623,$624,$625,$626,$627,$628,$629,$630),($631,$632,$633,$634,$635,$636,$637,$638,$639,$640),($641,$642,$643,$644,$645,$646,$647,$648,$649,$650),($651,$652,$653,$654,$655,$656,$657,$658,$659,$660),($661,$662,$663,$664,$665,$666,$667,$668,$669,$670),($671,$672,$673,$674,$675,$676,$677,$678,$679,$680),($681,$682,$683,$684,$685,$686,$687,$688,$689,$690),($691,$692,$693,$694,$695,$696,$697,$698,$699,$700),($701,$702,$703,$704,$705,$706,$707,$708,$709,$710),($711,$712,$713,$714,$715,$716,$717,$718,$719,$720),($721,$722,$723,$724,$725,$726,$727,$728,$729,$730),($731,$732,$733,$734,$735,$736,$737,$738,$739,$740),($741,$742,$743,$744,$745,$746,$747,$748,$749,$750),($751,$752,$753,$754,$755,$756,$757,$758,$759,$760),($761,$762,$763,$764,$765,$766,$767,$768,$769,$770),($771,$772,$773,$774,$775,$776,$777,$778,$779,$780),($781,$782,$783,$784,$785,$786,$787,$788,$789,$790),($791,$792,$793,$794,$795,$796,$797,$798,$799,$800),($801,$802,$803,$804,$805,$806,$807,$808,$809,$810),($811,$812,$813,$814,$815,$816,$817,$818,$819,$820),($821,$822,$823,$824,$825,$826,$827,$828,$829,$830),($831,$832,$833,$834,$835,$836,$837,$838,$839,$840),($841,$842,$843,$844,$845,$846,$847,$848,$849,$850),($851,$852,$853,$854,$855,$856,$857,$858,$859,$860),($861,$862,$863,$864,$865,$866,$867,$868,$869,$870),($871,$872,$873,$874,$875,$876,$877,$878,$879,$880),($881,$882,$883,$884,$885,$886,$887,$888,$889,$890),($891,$892,$893,$894,$895,$896,$897,$898,$899,$900),($901,$902,$903,$904,$905,$906,$907,$908,$909,$910),($911,$912,$913,$914,$915,$916,$917,$918,$919,$920),($921,$922,$923,$924,$925,$926,$927,$928,$929,$930),($931,$932,$933,$934,$935,$936,$937,$938,$939,$940),($941,$942,$943,$944,$945,$946,$947,$948,$949,$950),($951,$952,$953,$954,$955,$956,$957,$958,$959,$960),($961,$962,$963,$964,$965,$966,$967,$968,$969,$970),($971,$972,$973,$974,$975,$976,$977,$978,$979,$980),($981,$982,$983,$984,$985,$986,$987,$988,$989,$990),($991,$992,$993,$994,$995,$996,$997,$998,$999,$1000),($1001,$1002,$1003,$1004,$1005,$1006,$1007,$1008,$1009,$1010),($1011,$1012,$1013,$1014,$1015,$1016,$1017,$1018,$1019,$1020),($1021,$1022,$1023,$1024,$1025,$1026,$1027,$1028,$1029,$1030),($1031,$1032,$1033,$1034,$1035,$1036,$1037,$1038,$1039,$1040),($1041,$1042,$1043,$1044,$1045,$1046,$1047,$1048,$1049,$1050),($1051,$1052,$1053,$1054,$1055,$1056,$1057,$1058,$1059,$1060),($1061,$1062,$1063,$1064,$1065,$1066,$1067,$1068,$1069,$1070),($1071,$1072,$1073,$1074,$1075,$1076,$1077,$1078,$1079,$1080),($1081,$1082,$1083,$1084,$1085,$1086,$1087,$1088,$1089,$1090),($1091,$1092,$1093,$1094,$1095,$1096,$1097,$1098,$1099,$1100),($1101,$1102,$1103,$1104,$1105,$1106,$1107,$1108,$1109,$1110),($1111,$1112,$1113,$1114,$1115,$1116,$1117,$1118,$1119,$1120),($1121,$1122,$1123,$1124,$1125,$1126,$1127,$1128,$1129,$1130),($1131,$1132,$1133,$1134,$1135,$1136,$1137,$1138,$1139,$1140),($1141,$1142,$1143,$1144,$1145,$1146,$1147,$1148,$1149,$1150),($1151,$1152,$1153,$1154,$1155,$1156,$1157,$1158,$1159,$1160),($1161,$1162,$1163,$1164,$1165,$1166,$1167,$1168,$1169,$1170),($1171,$1172,$1173,$1174,$1175,$1176,$1177,$1178,$1179,$1180),($1181,$1182,$1183,$1184,$1185,$1186,$1187,$1188,$1189,$1190),($1191,$1192,$1193,$1194,$1195,$1196,$1197,$1198,$1199,$1200),($1201,$1202,$1203,$1204,$1205,$1206,$1207,$1208,$1209,$1210),($1211,$1212,$1213,$1214,$1215,$1216,$1217,$1218,$1219,$1220),($1221,$1222,$1223,$1224,$1225,$1226,$1227,$1228,$1229,$1230),($1231,$1232,$1233,$1234,$1235,$1236,$1237,$1238,$1239,$1240),($1241,$1242,$1243,$1244,$1245,$1246,$1247,$1248,$1249,$1250),($1251,$1252,$1253,$1254,$1255,$1256,$1257,$1258,$1259,$1260),($1261,$1262,$1263,$1264,$1265,$1266,$1267,$1268,$1269,$1270),($1271,$1272,$1273,$1274,$1275,$1276,$1277,$1278,$1279,$1280),($1281,$1282,$1283,$1284,$1285,$1286,$1287,$1288,$1289,$1290),($1291,$1292,$1293,$1294,$1295,$1296,$1297,$1298,$1299,$1300),($1301,$1302,$1303,$1304,$1305,$1306,$1307,$1308,$1309,$1310),($1311,$1312,$1313,$1314,$1315,$1316,$1317,$1318,$1319,$1320),($1321,$1322,$1323,$1324,$1325,$1326,$1327,$1328,$1329,$1330),($1331,$1332,$1333,$1334,$1335,$1336,$1337,$1338,$1339,$1340),($1341,$1342,$1343,$1344,$1345,$1346,$1347,$1348,$1349,$1350),($1351,$1352,$1353,$1354,$1355,$1356,$1357,$1358,$1359,$1360),($1361,$1362,$1363,$1364,$1365,$1366,$1367,$1368,$1369,$1370),($1371,$1372,$1373,$1374,$1375,$1376,$1377,$1378,$1379,$1380),($1381,$1382,$1383,$1384,$1385,$1386,$1387,$1388,$1389,$1390),($1391,$1392,$1393,$1394,$1395,$1396,$1397,$1398,$1399,$1400),($1401,$1402,$1403,$1404,$1405,$1406,$1407,$1408,$1409,$1410),($1411,$1412,$1413,$1414,$1415,$1416,$1417,$1418,$1419,$1420),($1421,$1422,$1423,$1424,$1425,$1426,$1427,$1428,$1429,$1430),($1431,$1432,$1433,$1434,$1435,$1436,$1437,$1438,$1439,$1440),($1441,$1442,$1443,$1444,$1445,$1446,$1447,$1448,$1449,$1450),($1451,$1452,$1453,$1454,$1455,$1456,$1457,$1458,$1 (truncated)⋅
the last thing is to mark all existing inputs as spent:


Add more i960 docs for MC, KB, and MX targets

Added Protected Architecture, Numerics Architecture, and Extended Architecture docs. This time with better author :).

Thanks!

I have never used the i960... a least knowingly.  Do you know about any interesting applications?  (Don't say printers.)

Printers would be the a29k actually :). I know they are the primary CPU of
Sega's Model 2 arcade board. They are also used in slot machines too. Also!
They are used in raid cards
On Thu, Sep 6, 2018, 23:09 Lars Brinkhoff<EMAIL_ADDRESS>wrote:

I have never used the i960... a least knowingly. Do you know about any
interesting applications? (Don't say printers.)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/larsbrinkhoff/awesome-cpus/pull/29#issuecomment-419333431,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABfnUk6GQaMKQOgmKPaWF3EI3LVuNIKcks5uYg2lgaJpZM4WeOzr
.


Question: how can you escape double quotes in search queries?

I have a query containing double quotes, e.g. a measurement in inches - '36"'. Right now I'm using tantivy.Index.query_parser(query) to parse the query. I have two questions:

What is the default behavior of the analyzer in this case for double quotes? (can probably answer myself with a bit more digging)
I haven't been able to find a way to parse this without hitting SyntaxError, is this a bug?

---> 21     query = self._index.parse_query(q.query)  # Search all fields
     22     search_results = self._tantivy_results_to_docs(
     23         self._searcher.search(query, q.limit).hits
     24     )
     25     return [
     26         SearchResultFromSystem(
     27             result=SearchResult(query=q, result=self._tantivy_doc_to_dict(doc)),
   (...)
     32         for idx, (score, doc) in enumerate(search_results)
     33     ]

ValueError: Syntax Error: fawkes 36" blue vanity

The approaches I've tried, all of which result in the same error above:
Escape the double quote: 36\"
Double or triple escape the quote: 36\\\" or 36\\"
Encode: '36"'.encode('utf-8')
Unicode from here: "36\\u{FF02}"
Raw python string: r'36"'
Raw python string w/ escapes from above: r'36\"'...
What should I do to get Tantivy to interpret this not as a field search, but as a literal double quote?

Hi @JLHasson, thanks for the report!
Could you make a tiny self-contained example we can run to reproduce the error? A simple approach could be to adapt one of the unit tests. Please also report the version of tantivy you're using test. I found problems with escaping quotes back in version 0.19.2 but I don't think I've seen those problems since. A simple reproducer we can just copy-paste and run would save time.

Hey @cjrh ! Sure thing:
$ poetry show | grep tantivy
tantivy                   0.21.0

Code:
import tantivy
query = '36"'
doc = {"title": 'some title with 36"'}
schema_builder = tantivy.SchemaBuilder()
schema_builder.add_text_field("title", stored=True)
_schema = schema_builder.build()
_index = tantivy.Index(_schema)
writer = _index.writer()
writer.add_document(tantivy.Document(**doc))
writer.commit()

_index.reload()
query = _index.parse_query(query)  # Search all fields
# ---> [13](vscode-notebook-cell:?execution_count=34&line=13) query = _index.parse_query(query)  # Search all fields
#
# ValueError: Syntax Error: 36"

Thank you for the reproducer, that helps.
It seems that the issue is the default tokenizer being used for the text field. If you change the tokenizer to raw or en_stem, the parse_query() call no longer fails.  raw is not super useful for general text fields, but hopefully you can work with en_stem.
import tantivy
query = r'title:"36\""'
doc = {"title": r'some title with 36\"'}

schema_builder = tantivy.SchemaBuilder()
schema_builder.add_text_field("title", stored=True, tokenizer_name="en_stem")
_schema = schema_builder.build()
_index = tantivy.Index(_schema)
writer = _index.writer()
writer.add_document(tantivy.Document(**doc))
writer.commit()

_index.reload()
query = _index.parse_query(query)  # Search all fields
print(query)

searcher = _index.searcher()
results = searcher.search(query, 3).hits
print(results)

Note that the en_stem tokenizer will be stemming the tokens, which may not be what you want.  This is the output that is produced running the above:
$ python main.py
Query(TermQuery(Term(field=0, type=Str, "36")))
[(0.28768211603164673, <tantivy.DocAddress object at 0x71eb645fdf70>)]

You can see that the escaped quote in the query is lost after parsing. The r"36\"" becomes just 36.
This seems like a bug in the default tokenizer.

For your specific use-case, it may be necessary to make your own tokenizer.

Hmm, I was wrong about the default tokenizer having a bug, it seems the same escaping works as I described above in my en_stem example:
import tantivy
query = r'title:"36\""'
doc = {"title": r'some title with 36"'}

schema_builder = tantivy.SchemaBuilder()
schema_builder.add_text_field("title", stored=True, tokenizer_name="default")
schema = schema_builder.build()
index = tantivy.Index(schema)
writer = index.writer()
writer.add_document(tantivy.Document(**doc))
writer.commit()

index.reload()
query = index.parse_query(query)  # Search all fields
print(query)
print(index.searcher().search(query, 1).hits)

Again though, the extra " is lost after parsing because the tokenizer rules in default drop it.