Feature request: Specify landing page for root directory

Specify a default landing page for root directory, retain showDir: true everywhere else.
Perhaps rootLanding: 'html/home.html' as an example.
Currently the landing page serves the auto-generated index of files.
(not sure if this should be requested on indexzero/http-server instead?)

Hi @JKAussieSkater, the grunt tasks just passes the showDir parameter to http-server module, which in it's turn is passing that to ecstatic module.
From what I can see on https://github.com/jfhbrook/node-ecstatic/blob/master/lib/ecstatic/showdir.js that's
not possible.
The alternative would be to set the before http-server callback to redirect to another page, e.g. /home.html
Is there a particular use case for this?

Sorry, I didn't explain myself properly.
When I navigate to http://<IP_ADDRESS>:8282/ I want it to display a "homepage.html".
Instead I get the contents of the root: '/foo/bar/' directory.
Is there an option to specify a homepage, so I don't have to navigate a directory file-tree?

Let me get back to you on that. I think the before middleware callback can be used for that.

Hi @JKAussieSkater I've just added the support to customPages that will help you with your landing page. Please take a look at the README.md.
Help the project by donating and spreading the word.
Let me know if you have any questions.

@aetheon, you're a super-human!
Thank you for this, I will definitely spread the word! So many will love this new feature!
I attempted to donate, but the link didn't work:

No worries @JKAussieSkater glad to help.


Memory leaks in ClosureEvents?

[important!]
I was using pre1.29 version, so I updated ClosureEvents but this issue didn't disappaer. This may need to be checked in recent lib.
===============================================================================
Let's create an empty map and try compiling these codes.
[must needed]
package TestCode
import ClosureTimers

init
	doPeriodically(0.01) (CallbackPeriodic cb) ->
		createUnit(Player(0), 'hgry', vec2(0,0), angle(0))
		..hide()
		..setTimedLife(0.1)

[optional]
This is simply made for detecting leaks caused by 'handles'. You can use other code you have for debugging.

package HandleProfiler
import ClosureTimers

function getNewOffset() returns int
	let g = CreateGroup()
	let id = g.getHandleId()
	g.destr()
	return id - 0x100000

class ProfilerVars
	static int maxOffset

init
	doAfter(1.0) ->
		let offset = getNewOffset()
		ProfilerVars.maxOffset = offset
		doPeriodically(1.0) (CallbackPeriodic cb) ->
			let newOffset = getNewOffset()
			if newOffset > ProfilerVars.maxOffset
				let increase = newOffset - ProfilerVars.maxOffset
				var str = "max handle offset increased from "
				str += ProfilerVars.maxOffset.toString() + " to "
				str += newOffset.toString() + ", +" + increase.toString()
				print(str)
				ProfilerVars.maxOffset = newOffset

If you run these code, you will see the message that max handle offset is increasing more and more. This means that memory is leaked by something.
I found that leaking is caused in ClosureEvents.
// Un/Register Events when unit enters map
onUnitIndex(() -> registerEventsForUnit(getIndexingUnit()))
onUnitDeindex(() -> unregisterEventsForUnit(getIndexingUnit()))

if you edit this,
// Un/Register Events when unit enters map
onUnitIndex(null)
onUnitDeindex(null)

You can see that max handle is no more inflating.
#note! Because of war3 internal problem, making a unit always leak. So also in this case, you can see the war3's memory increasing in task manager
Things that always leak

What's the problem? Seems expected for handle counter to increase when events are registered.
But neither handle counter nor windows memory usage haven't been reliable measures for a while now, both of them recycle/reset.
I have run maps extensively using closure events with no problems for hours, so I don't think it matters.

Thanks for the reply.
There is one thing I want to check first.
Can you see that the maximum handle value goes up when you run the code [part1] as I wrote in the post?

Considering your comments, I tried a few things. And I came to the following conclusions:

I tested the [part1] code for a long time and it didn't encounter fatal as quickly as I expected.
On the contrary, the game ran fine for a much much longer than I expected. (I faced a fatality when the max handle went over 922500.)
So, as you said, I haven't identified any significant problems.

As you mentioned, the method of detecting memory leaks by increasing the maximum handle value is not reliable.

What is the memory leak detection method you recommend?

Below are about how I reached these conclusions.
You said,
"neither handle counter nor windows memory usage have been reliable measures"
It's the opposite of what I've been taught for a long time.
When I was in the early days of map production, I had a game crash (fatal) due to an effect leak.
Since then, I've been aware of memory leaks and have seen several posts trying to get rid of them by designating them as absolute evil.
And I came up with a way to check for memory leaks by updating the max handle value.
I also used this to catch a lot of leaks when I was using vjass .
So what you said was a little questionable, and I decided to experiment.
The content of the experiment is to measure the point at which the game suffers a fatality and is forcibly terminated when the maximum handle value is reached if the above [part1] is continuously executed.
As a result.
I faced a fatal when [max handle value 923000, memory usage 1.1GB]
That's right. It is 923000. It's not the 92300. This means that the [part1] code will have to create more than 900,000 (...) units during the game to suffer a memory failure.
This was far beyond my expectations. In the early days of map making, I suffered a fatality even when there were over a thousand effect leaks, but in the case of the code above, the game runs fine even when it reaches 900,000.
And above all, right before the fatal, there were no frame drops in the game.
This experiment made me realize that leak detection methods that measure the maximum handle value increase are unreliable, and handle increases caused by ClosureEvents are also fine.

It's the opposite of what I've been taught for a long time.

It was true for earlier patches I suppose. Thanks for posting your results.
The main takeaway imho is that you should worry about performance issues when you face them and not try to optimize prematurely.


remove jsk_perception due to failing source debs

See: http://build.ros.org/job/Jsrc_uU__jsk_perception__ubuntu_utopic__source/20/
These jobs would fail every 15 minutes. @k-okada I'm going to merge this now to stop them from failing continuously. You can create a new release when you think you've addressed the issue.

Why not revert to the previous version? This would remove all jobs as well as downstream jobs.

Oh I did not know it is my fault, now I understand the reasons and will fix as soon as I arrived the office in 15 mins
--
◉ Kei Okada
2016/02/09 8:58、William Woodall<EMAIL_ADDRESS>のメッセージ:

See: http://build.ros.org/job/Jsrc_uU__jsk_perception__ubuntu_utopic__source/20/
These jobs would fail every 15 minutes. @k-okada I'm going to merge this now to stop them from failing continuously. You can create a new release when you think you've addressed the issue.
You can view, comment on, or merge this pull request online at:
https://github.com/ros/rosdistro/pull/10446
Commit Summary
remove jsk_perception due to failing source debs
File Changes
M jade/distribution.yaml (15)
Patch Links:
https://github.com/ros/rosdistro/pull/10446.patch
https://github.com/ros/rosdistro/pull/10446.diff
—
Reply to this email directly or view it on GitHub.

I didn't notice there was an intermediate state which built: https://github.com/ros/rosdistro/pull/10447

@k-okada That's fine, take your time, I'm just going to revert it in the mean time.

Sorry it takes more than 15min, (wait for travis,,,,)  https://github.com/ros/rosdistro/pull/10448 will fix this


Return interface types from To[ReadOnly]ReactiveProperty()

Thank you, nice improvement and sorry, I've changed directory structure so can't merged.
I've added same changes.

Ah, I've reverted ToReactiveProeprty returns ReactiveProperty.
It has some reasons.

breaking change
ToReactiveProperty is recommend way for create ReactiveProperty, and we have to provide full ReactiveProperty feature(SetValueAndForceNotify, Dispose)


Add results variable definition

Have this somewhere recorded:

https://github.com/danger/danger-js/issues/1253#issuecomment-1074397856

at least new plugins then?

Cool, looks good to me 👍🏻  thanks


use here for trade areas

cc @saleiva
CR: @xavijam

That's it?

Please, let's check the error

that's from the frontend side, the error depends on backend


Ethics & Responsibility

I understand might be an interesting experiment in some context. But I have huge ethical concerns. By doing this you basically support and amplify the ( IMHO already way to high ) power of trump. Do you really want this? I think this app should come with a big red warning sign - I would even be a bit more radical as greedy people might still use it despite this warning sign.

This is Github not politicsub.

@Pablocgdev  I am on github way longer than you - with way more contributions/repos/stars - so please do not tell me what this platform is about!
I am really sad about this repository and the reactions - really hope some of you broaden this very dangerous and narrow tunnel-vision you are into currently.

Oh, I forgot the number of stars, aging and contributions makes your arguments stronger.
Nothing to talk with you. The only vision-tunnel here is yours, trying to censor a software because you don't like the politic view behind it, I'm sure a lot of people here doesn't share Trump politics but focus on development.

I do not want to censor anything - but I just think it is really important to reflect on the implications of what you are working on. The humans building the gas-chambers where also only focusing on development and did not care about politics ..
And github is the place to discuss this - this is social coding - not asocial coding.

Just stumbled upon this repository and issue. I find it hilarious:

The fact the repo exists in the first place is just pure gold.
The no. 1 issue is called "Ethics & Responsibility"
The issue is actually meant to be taken seriously (like sane people would trust a bot that parses natural language - and that of the Orange One nonetheless - with their savings; have you looked at the picture in the readme?)
There are people arguing about this
One logical fallacy after the other ("stars make your arguments stronger" - thank you for this, "a little stock-trading app is like nazi gas chambers", "censorship is when people voice their opinions and ASK rather POLITELY to consider taking the repo down")

They say good comedy is when you can just barely tell if it's a joke. Are you guys serious?

I really hope @Pablocgdev is joking - and also really hope the author is joking - perhaps even some nasty joke like with some undocumented feature that transfer all funds to some environment protection NGO from people that actually are so greedy to use this software.
But I cannot speak for others - I for myself am not joking ( here currently - sometimes I am - but this here now is serious )

A regular Cuban Missile Crisis.

Lighten up. This is hilarious. I for one will be pumping the millions I earn from this tool into my Pro-Trump charity for "People Who Support Trump and Want to Learn To Do Other Stuff Good Too". I look forward to committing to this repo to help "raise the profile" of that little unknown loudmouth called THE PRESIDENT OF THE UNITED STATES.

I still can't tell if @ligi is joking or not. But maybe you should read the repo author's article which suggests donating any profits to charities such as the ACLU, Planned Parenthood, etc.
Hopefully that will unrustle those jimmies.

Donating the profits to charities is not really helping. It is the same as selling someone a weapon to kill someone else and then giving the profit to the widow..

Straw man

Hey man after I discovered all of this I accidentally broke my phone and
had no way to continue with my page, so that is why I haven't responded. I
hope everything is under control. What is control?
On Mar 1, 2017 8:45 PM, "Colm Seale"<EMAIL_ADDRESS>wrote:

Lighten up. This is hilarious. I for one will be pumping the millions I
earn from this tool into my Pro-Trump charity for "People Who Support Trump
and Want to Learn To Do Other Stuff Good Too". I look forward to committing
to this repo to help "raise the profile" of that little unknown loudmouth
called THE PRESIDENT OF THE UNITED STATES.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/maxbbraun/trump2cash/issues/37#issuecomment-283530333,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYh1HL_Ukb-IDT3FRDfrMg5ycq_2oZ2gks5rhh89gaJpZM4MFdg9
.

This aged well

I for one am really happy twitter solved the problem and this guy is neither on twitter anymore nor POTUS ..
And once more disappointed in google. Seems Google's 'Don't Be Evil' was a canary seeing googlers behind repositories like this.


KafkaJs consumer taking longer time if keeping groupID same

Hi Team,
we are facing the same issue, however time remain unpredictable.
As we are using the same code KafkaJS in the production environment, so this is not possible to change the consumer groupID after each call, it must kept same to make make sure that we can read only those message which was not read from the groupID, if we keep changing the groupID on each request/interval then we will always getting the all message again and again, yes this improvement connection and message getting time like anytime.
Current Stat
Keeping same consumer group ID: 20-30 Seconds
Keep changing the groupID: 2-5 mili second
here we are looking for your help, how to improve the connection time and message fetching keeping consumer groupID same
my code base is.....
.env
DEFAULT_MESSAGE_POLLING_TIME=700
KAFAK_BROKER_ENDPOINT=localhost:9094
KAFAK_OUTBOUND_TOPIC_PREFIX=
KAFAK_OUTBOUND_TOPIC_POSTFIX=_OUT
I have set the consumer group name and topic name in the Database, in this case
consumer_group_name = "customer_demo_group"
outbound_topic = "customer_demo_OUT"
/***************************************************************************/
/*Purpose   : Polling the mesage from topic.
/*Input     : None.
/*Response  : None.
/*Created By: Jaiswar Vipin Kumar R.
/***************************************************************************/
setInterval(function(){
	/* Iterating the operation array loop */
	Object.keys(strTopicArr).forEach(function(key, index) {
		//console.log(this[key].inbound_topic);
		/* get the message of each outbound topic */
		resetTheConsumer(this[key].consumer_group_name, this[key].outbound_topic);
	}, strTopicArr);
},process.env.DEFAULT_MESSAGE_POLLING_TIME);

and consumer code base is
/***************************************************************************/
/*Purpose   : Reading Message from topics.
/*Input     : pStrConsumerGroup :: Topic consumer Group,
			: pStrTopicName :: Topic name.
/*Response  : Message Detail
/*Created By: Jaiswar Vipin Kumar R.
/***************************************************************************/
function getMessageFromTopic(pStrConsumerGroup, pStrTopicName, callback){
	try{
		/* Getting the kafak avaliable messge broker reference */
		const { Kafka } 				= require('kafkajs')
		const kafakBrokerendPointArr 	= (process.env.KAFAK_BROKER_ENDPOINT).split(',');
		const kafka 					= new Kafka({clientId: pStrConsumerGroup+'_client',brokers: kafakBrokerendPointArr,logLevel:0});

		/* Creating the consumer */
		//const consumerObj = kafka.consumer({ groupId: pStrConsumerGroup+"_"+getRandomInt(10)});
		const consumerObj = kafka.consumer({ groupId: pStrConsumerGroup});

			/* Creating the non-blocking aync consumer request */
			const runs = async () => {
				/* Connecting to the consumer */
				consumerObj.connect()
				.then(returnObj =>{
					/* Subscribing consumer to the topic */
					consumerObj.subscribe({ topic: process.env.KAFAK_OUTBOUND_TOPIC_PREFIX+pStrTopicName+process.env.KAFAK_OUTBOUND_TOPIC_POSTFIX, fromBeginning: true})
					//console.log(pStrConsumerGroup+" === "+pStrTopicName);
					/* Runnign the look for getting the messge */
					consumerObj.run({
						partitionsConsumedConcurrently: 3,
						eachMessage: async ({ topic, partition, message }) => {
							try{
								//console.log(message.value.toString());
								if(message.value.toString() !=''){
									/* return callback({
										partition,
										offset: message.offset,
										value: message.value.toString(),
									}) */
									//console.log(message.value.toString());
									sendMessage(message.value.toString());
								}
							}catch(Exception){
								console.log(Exception);
							}

						}
					})
				})
				.catch(excp =>{
					console.log(excp);
				}).finally(fObj=>{

				})
				consumerObj.disconnect();
			}
			runs().catch(e => console.error(`[example/consumer] ${e.message}`, e))

	}catch(exception){
		console.log(exception);
	}
}

and we expecting 0.1-0.2 billions of message in the 15 min time to be get consumer and send to the requester
@tulios  @Nevon  Kindly assist
Originally posted by @jaiswarvipin in https://github.com/tulios/kafkajs/issues/260#issuecomment-654489802

setInterval(() => { ... resetTheConsumer(...).then(...) }, process.env.DEFAULT_MESSAGE_POLLING_TIME)

I don't see that you posted your resetTheConsumer function, but the name is very very suspicious: Are you really reconnecting each time? Kafka uses connections and long-polling to be efficient. This has also effects on how consumer groups work, and there are timeouts that Kafka must obey to know all members of a consumer group. You should read up a bit on that topic.

		const kafakBrokerendPointArr 	= (process.env.KAFAK_BROKER_ENDPOINT).split(',');
					consumerObj.subscribe({ topic: process.env.KAFAK_OUTBOUND_TOPIC_PREFIX+pStrTopicName+process.env.KAFAK_OUTBOUND_TOPIC_POSTFIX, fromBeginning: true})

Typos: It's "kafka", not "kafak" :)

https://github.com/tulios/kafkajs/issues/701 probably related. Most likely the reason for slow connection is due to session timeout as if you disconnect from CG it may take some time for server to expire your previous connection. Also i noticed that consumerObj.disconnect(); is not await. I think kafka is better used with single consistent connection rather then periodic re connection.

Thank @ankon  and @goriunov  for you view on the code. Yes i have corrected it and now now calling the consumer on-time only and that's help and now my response time for first and followed message is
Current Stat
Keeping same consumer group ID: 3-6 mili second
@ankon resetTheConsumer is just the method who calling the getMessageFromTopic method and thanks for spell correction :)
However, needed your view on the procedure as well.
Allow me to give background: currently we are doing the testing the with 10K message in 3 seconds (as of now) and it will go till the 0.1 billions...
The code base for procedure is ( currently procedure creating the inbound and outbound topic automatically such that other side scheduler will pick the message from inbound topic and put response from into outbound topic and from where our consumer (above getMessageFromTopic method) is consuming the message and sending back to the requester and its job to just out the incoming message into the inbound topic)
/***************************************************************************/
    /*Purpose   : Send Message into the topics.
    /*Input     : pStrConsumerGroup :: Consumer Group name,
				: pStrTopicName :: Topic name,
				: pStrOutBoundTopicName :: Outbound topic name.
                : pStrMessageBody   :: Message body
    /*Response  : Message send status :: TRUE/FALSE
    /*Created By: Jaiswar Vipin Kumar R.
    /***************************************************************************/
    setMessageInTopic(pStrConsumerGroup, pStrTopicName,pStrOutBoundTopicName, pStrMessageBody, callback){
		try{
			//console.log(pStrConsumerGroup+" === "+pStrTopicName+" === "+pStrOutBoundTopicName);
			/* Variable initilization */
			const { Kafka } 				= require('kafkajs')
			const kafakBrokerendPointArr 	= (process.env.KAFAK_BROKER_ENDPOINT).split(',');
			const kafka 					= new Kafka({clientId: pStrConsumerGroup+'_client',brokers: kafakBrokerendPointArr,logLevel:0});
			const producer 					= kafka.producer()

			/* Creating the non-blocking aync prodecure request */
			const run = async () => {
				try{
					/* Creating teh topic and sending the message  */
					await producer.send(
										{
											topic: pStrTopicName,
											messages: [
														{
															value: JSON.stringify(pStrMessageBody)
														},
													],
										}
									);

					/* Creating the outbound topic */
					await producer.send(
										{
											topic: process.env.KAFAK_OUTBOUND_TOPIC_PREFIX+pStrOutBoundTopicName+process.env.KAFAK_OUTBOUND_TOPIC_POSTFIX,
											messages: [
														{
															value: ""
														},
													],
										}
									);
					/* Return the response */
					return callback(true);
				}catch(exception){
					/* Print the exception */
					console.log(exception);
					/* Return the response */
					return callback(false);
				}
			}
			/* Throw if any execption occured while performing the procedure operation */
			run().catch(console.error);
		}catch(exception){
			/* Print the exception */
			console.log(exception);
			/* Return the response */
			return callback(false);
		}
	}

one observation we found when we are start sending the 10K+ message in 3 seconds at few (30%) message are available after 1 min and because of same the TAT getting impacted.
Kindly suggest
Thanks.

I have the same problem when using nodemon for debugging. Every time I edit a file, nodemon will restart my program. I add consumer.disconnect() like this:
  const signalTraps = ['SIGTERM', 'SIGINT', 'SIGUSR2']

  signalTraps.map(type =>
        process.once(type, async () => {
          await this.consumer.disconnect();
          logger.info("kafka consumer disconnect");
        })
      );
    }

And when it restarts, it outputs:

[start:server] [2021-03-01T08:48:39.349+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.349+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.349+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.349+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.349+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.349+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.350+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.350+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.350+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.350+0100] [INFO] service - kafka consumer disconnect
[start:server] [2021-03-01T08:48:39.350+0100] [INFO] service - kafka consumer disconnect

It shows that consumers indeed disconnected. But consumers still taking a long time to connect if keeping the same groupID, exactly it seems like the sessionTimeout value. Changing groupID is not a graceful idea. It will be better if I set
heartbeatInterval: 2000,
sessionTimeout: 6000

It doesn't occur if I use a different groupID or use Ctrl+C to exit the process and restart it manually, so I wonder why reconnecting should take  sessionTimeout time with nodemon, it should be the same with node. How can I avoid such a long time?

What you need to keep in mind is that the consumer group exists outside of your node instances. Whenever a consumer joins or leaves the group, all the members of the group have to re-join and sync before they can start consuming again.
When you start your node process, your consumer will join that consumer group, and when you quit the process (with graceful disconnect), your consumer will leave the consumer group. Like mentioned above, this means the group has to rebalance (re-join and sync).
When you just exit the process without disconnecting, your consumer doesn't actually leave the group - so initially there won't be a rebalance. However, after sessionTimeout that consumer will be considered unhealthy and will be kicked out of the group, which will trigger a rebalance. In the in-between time, no processing will happen on any partitions that are assigned to that consumer.
If you are developing with nodemon, causing frequent restarts, a solution is probably to generate a random groupid on each restart. When static membership (#884) becomes available, that could be another option, but it's not available yet so for now a random group id is probably the best bet. Either that, or you have to wait for the rebalance to happen.

Thanks for your explanation~ I still have a question, I add consumer.disconnect in the code, and my consumers should disconnect when the process restarts with nodemon. Is it normal for rebalance to take such a long time? Whether will consumer.disconnect work when developing with nodemon...

Is it normal for rebalance to take such a long time? Whether will consumer.disconnect work when developing with nodemon

It will take up to rebalanceTimeoutms if you don't gracefully disconnect.
From the documentation:

rebalanceTimeout
The maximum time that the coordinator will wait for each member to rejoin when rebalancing the group
default: 60000

yeah, I get it, how about I set this value to less than 10000, any side effects will be made? I know this option but no detailed description in the documentation.
Thanks a lot! 🙇

It's pretty important that you understand how heartbeatInterval, sessionTimeout and rebalanceTimeout fit together. If rebalanceTimeout is less than heartbeatInterval, for example, it would mean that if you have multiple consumers in the group, when a new instance joins and triggers a rebalance, the other consumers might get kicked out of the group because they don't become aware that there is a rebalance going on until they heartbeat.
This talk https://www.confluent.io/online-talks/everything-you-always-wanted-to-know-about-kafkas-rebalance-protocol-but-were-afraid-to-ask-on-demand explains everything very well. There are also loads of other resources on the internet explaining this.

Is it normal for rebalance to take such a long time? Whether will consumer.disconnect work when developing with nodemon

It will take up to rebalanceTimeoutms if you don't gracefully disconnect.
From the documentation:

rebalanceTimeout
The maximum time that the coordinator will wait for each member to rejoin when rebalancing the group
default: 60000

Hi Nevon,
After several weeks I'm working on this problem again, since random group id will bring to some questions we don't expect. eg. with different consumer group id, our new consumer may consume the same message again that the old consumer has consumed.
I am confused about gracefully disconnect, currently I just have this piece of code when exiting:
  const signalTraps = ['SIGTERM', 'SIGINT', 'SIGUSR2']

  signalTraps.map(type => {
    process.on(type, async () => {
      logger.info(`Kafka client: process prepare to exit.`)

      await Promise.all(this.allConsumer.map(
        async (consumer)=>{
          await consumer.disconnect();
        }
      ));
      await this.producer.disconnect();
      logger.info(`Kafka client: producer disconnected`);
      await this.admin.disconnect();
      logger.info(`Kafka client admin disconnected`);
      process.exit(0);
    })
  });

It shows in the log that the consumer indeed stopped:
[2021-03-22T08:42:49.886+0100] [INFO] service - Kafka client: process prepare to exit.
{"level":"INFO","timestamp":"2021-03-22T07:42:49.887Z","logger":"kafkajs","message":"[Consumer] Stopped","groupId":"Monitor"}
{"level":"INFO","timestamp":"2021-03-22T07:42:51.285Z","logger":"kafkajs","message":"[Consumer] Stopped","groupId":"Monitor"}
[2021-03-22T08:42:52.345+0100] [INFO] service - Kafka client: producer disconnected
[2021-03-22T08:42:52.346+0100] [INFO] service - Kafka client admin disconnected

Is this graceful disconnecting? But in the Kafka server, I still see that the consumer was kicked out after session timeout.
[2021-03-22 07:43:01,282] INFO [GroupCoordinator 0]: Member kafkajs-2d4e3bd3-d7ca-41bd-b1e6-05a4bb5d1210 in group Monitor has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)
[2021-03-22 07:43:01,282] INFO [GroupCoordinator 0]: Preparing to rebalance group Monitor in state PreparingRebalance with old generation 6 (__consumer_offsets-38) (reason: removing member kafkajs-2d4e3bd3-d7ca-41bd-b1e6-05a4bb5d1210 on heartbeat expiration) (kafka.coordinator.group.GroupCoordinator)
[2021-03-22 07:43:01,282] INFO [GroupCoordinator 0]: Group Monitor with generation 7 is now empty (__consumer_offsets-38) (kafka.coordinator.group.GroupCoordinator)

Does it mean that consumer.disconnect doesn't work?
Thanks in advance.

If you enable debug logs, you can see exactly what happens. What should be happening is that the consumer(s) wait for ongoing work, send a GROUP_LEAVE request and then disconnect.

Yes, I see it like this:
{"level":"INFO","timestamp":"2021-03-22T09:08:14.053Z","logger":"kafkajs","message":"[Consumer] Stopped","groupId":"Monitor"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:14.054Z","logger":"kafkajs","message":"[Consumer] consumer has stopped, disconnecting","groupId":"Monitor"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:14.054Z","logger":"kafkajs","message":"[Connection] disconnecting...","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:14.054Z","logger":"kafkajs","message":"[RequestQueue] Waiting for pending requests","clientId":"kafkajs","broker":"<IP_ADDRESS>:9092","currentInflightRequests":1,"currentPendingQueueSize":0}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.160Z","logger":"kafkajs","message":"[Runner] consumer not running, exiting","error":"Not connected","groupId":"Monitor","memberId":"kafkajs-527d1166-cdb4-4da4-97c8-b4d193e47623"}
{"level":"INFO","timestamp":"2021-03-22T09:08:17.160Z","logger":"kafkajs","message":"[Consumer] Stopped","groupId":"Monitor"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.160Z","logger":"kafkajs","message":"[Consumer] consumer has stopped, disconnecting","groupId":"Monitor"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.160Z","logger":"kafkajs","message":"[Connection] disconnecting...","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.160Z","logger":"kafkajs","message":"[Connection] disconnected","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.160Z","logger":"kafkajs","message":"[Connection] disconnecting...","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.160Z","logger":"kafkajs","message":"[Connection] disconnected","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.170Z","logger":"kafkajs","message":"[Connection] disconnected","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.170Z","logger":"kafkajs","message":"[Connection] disconnecting...","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.170Z","logger":"kafkajs","message":"[Connection] disconnected","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs"}
{"level":"DEBUG","timestamp":"2021-03-22T09:08:17.170Z","logger":"kafkajs","message":"[Connection] Response Fetch(key: 1, version: 11)","broker":"<IP_ADDRESS>:9092","clientId":"kafkajs","correlationId":25,"size":134,"data":"[filtered]"}

I wonder why the Kafka server still kicked out the consumer after session timeout, even if I disconnect the consumer when the process exits. From your explanation above, I think it may not normal.

I believe I am facing a similar issue to this. Is there any update on the fix? Based on #701 it seems that this is a known issue, and the workaround for now is to use different groupIds. Is that correct?

@sidyag I faced the same issue in e2e tests.
I had multiple test scenarios creating consumers with the same group id and despite calling consumer.disconnect() it was taking rebalanceTimeout to run a new consumer.
The solution that seems to be working for me is to create a new instance of Kafka for each test scenario to isolate consumers from one another.
I hope it helps.

Thank @ankon and @goriunov for you view on the code. Yes i have corrected it and now now calling the consumer on-time only and that's help and now my response time for first and followed message is
Current Stat Keeping same consumer group ID: 3-6 mili second
@ankon resetTheConsumer is just the method who calling the getMessageFromTopic method and thanks for spell correction :)
However, needed your view on the procedure as well. Allow me to give background: currently we are doing the testing the with 10K message in 3 seconds (as of now) and it will go till the 0.1 billions...
The code base for procedure is ( currently procedure creating the inbound and outbound topic automatically such that other side scheduler will pick the message from inbound topic and put response from into outbound topic and from where our consumer (above getMessageFromTopic method) is consuming the message and sending back to the requester and its job to just out the incoming message into the inbound topic)
/***************************************************************************/
    /*Purpose   : Send Message into the topics.
    /*Input     : pStrConsumerGroup :: Consumer Group name,
				: pStrTopicName :: Topic name,
				: pStrOutBoundTopicName :: Outbound topic name.
                : pStrMessageBody   :: Message body
    /*Response  : Message send status :: TRUE/FALSE
    /*Created By: Jaiswar Vipin Kumar R.
    /***************************************************************************/
    setMessageInTopic(pStrConsumerGroup, pStrTopicName,pStrOutBoundTopicName, pStrMessageBody, callback){
		try{
			//console.log(pStrConsumerGroup+" === "+pStrTopicName+" === "+pStrOutBoundTopicName);
			/* Variable initilization */
			const { Kafka } 				= require('kafkajs')
			const kafakBrokerendPointArr 	= (process.env.KAFAK_BROKER_ENDPOINT).split(',');
			const kafka 					= new Kafka({clientId: pStrConsumerGroup+'_client',brokers: kafakBrokerendPointArr,logLevel:0});
			const producer 					= kafka.producer()

			/* Creating the non-blocking aync prodecure request */
			const run = async () => {
				try{
					/* Creating teh topic and sending the message  */
					await producer.send(
										{
											topic: pStrTopicName,
											messages: [
														{
															value: JSON.stringify(pStrMessageBody)
														},
													],
										}
									);

					/* Creating the outbound topic */
					await producer.send(
										{
											topic: process.env.KAFAK_OUTBOUND_TOPIC_PREFIX+pStrOutBoundTopicName+process.env.KAFAK_OUTBOUND_TOPIC_POSTFIX,
											messages: [
														{
															value: ""
														},
													],
										}
									);
					/* Return the response */
					return callback(true);
				}catch(exception){
					/* Print the exception */
					console.log(exception);
					/* Return the response */
					return callback(false);
				}
			}
			/* Throw if any execption occured while performing the procedure operation */
			run().catch(console.error);
		}catch(exception){
			/* Print the exception */
			console.log(exception);
			/* Return the response */
			return callback(false);
		}
	}

one observation we found when we are start sending the 10K+ message in 3 seconds at few (30%) message are available after 1 min and because of same the TAT getting impacted.
Kindly suggest
Thanks.

Hi,
I didn't understand what did you change to reduce the consumer joining the group time ?

Have the same issue with the NestJs Kafka application, reconnect to kafka takes 15-30 sec when I'm using hot reload, this is really annoying while developing an app.

Have the same issue with the NestJs Kafka application, reconnect to Kafka takes 15-30 sec when I'm using hot reload, this is really annoying while developing an app. My app gracefully disconnects from Kafka and then connects...

So how did you solved?

Have the same issue with the NestJs Kafka application, reconnect to Kafka takes 15-30 sec when I'm using hot reload, this is really annoying while developing an app. My app gracefully disconnects from Kafka and then connects...

So how did you solved?

Unfortunately didn't find a solution yet :(


owserAuthError: block_nested_popups: Request was blocked inside a popup because MSAL detected it was running in a popup.

Core Library
MSAL.js v2 (@azure/msal-browser)
Core Library Version
2.21.0
Wrapper Library
Not Applicable
Wrapper Library Version
0.0
Description
I am authenticating using msal library, using PublicClientApplication

const myMsal = new PublicClientApplication(config);
const loginResponse = await myMsal.loginPopup(loginRequest)

It is authenticating correctly, I am getting token correctly. But in msal login popup, I get an error alert.
Error Message
BowserAuthError: block_nested_popups: Request was blocked inside a popup because MSAL detected it was running in a popup.
Msal Logs
No response
MSAL Configuration
const config = {
     auth: {
         clientId: 'bab9fad8-5de6-4076-9d59-4e6c0e88c00c',
         authority:
           'https://login.microsoftonline.com/8331e14a-9134-4288-bf5a-5e2c8412f074',
         redirectUri: `http://localhost:3000/oauth2/token`
       },
       cache: {
         cacheLocation: 'localStorage', // This configures where your cache will be stored
         storeAuthStateInCookie: true, // Set this to "true" if you are having issues on IE11 or Edge
       }
 }

> const loginRequest = {
>   scopes: ['openid'],
> }

Relevant Code Snippets
const myMsal = new PublicClientApplication(config);
const loginResponse = await myMsal.loginPopup(loginRequest)

Reproduction Steps
Download msal Library
and try running the authentication mechanism using the above code snippets and configuration
Expected Behavior
it should login without giving any error alert
Identity Provider
Azure AD / MSA
Browsers Affected (Select all that apply)
Chrome
Regression
No response
Source
External (Customer)

Screenshot of the error

@ankit-bagaria-kr This is likely happening because the page your app is using for the redirect uri (localhost:3000/oauth2/token) has code that is checking for the user's authentication state and invoking a popup method while MSAL is trying to process the initial popup operation. To mitigate this, make sure this code does not run while this page is in a popup.


UI Item confusion

What is the significance of
This is in the task bar of atom and I don't know what it means.

It means that YAPF checked your python and it isn't formatted to yapf's specifications.

maybe an alt/title tag would be in order here.


[FEATURE] command migration

Adds support for migrating the <command> element.  Ensures that it migrates into the new command_button element, which is an inline element.  Care is taken to ensure these always exist inside of a p or li or other block level parent.
While testing this out I discovered that video migration was broken.  embed_activity support was making <source> specific changes, which conflicts with the video element <source> element.

Closing, this work landed with a previous PR (that PR branched from this one)


Add option to start daemon without   schedules

Expected behaviour:
A option to start daemon without the schedules, this would allow using the api, or do some test, without having to comment out the schedules if they are set
Actual behaviour:
When we what to use the api, we have to start the daemon, with this the schedules are set, it would be nice to have a modifier to the damon to not start them. Like flexget daemon start -d --no-schedules
Would this be a big change? Is it possible?
Thank you

Hmmm, this is a very interesting idea. So the use case is that you want to have schedules but you also want the ability to start the WEBUI/API without triggering them, correct?

Basically yes. In my case, I have the schedules set, but If I what to test the api and not schedules, I have to comment out the schedules!
I noticed this because I tried to do a symbolic link from my config to start a new instance of the flexget with only the api, to do some experiments, but I didn't what the schedules to run. But I also wanted to use the same config file because I wanted to make changes and replicate them to the main instance. I also wanted to avoid copy paste, to minimize forgetting something and losing hours finding what was it!
So, like we do flexget daemon start -d, to start the daemon in background, we could have something like flexget daemon start -d --no-schedule, or flexget daemon start -ns,

First off, from a technical stand point, it is possible and doesnt look that painful to do.
That being said, I'm not sure it's worth it.
Flexget is an automator, one that supposed to constantly run and do... stuff. What is the downside of tasks running even when you just want to check out the API? What is the reason you want to disable the schedules?

The main reason is that I want to automate the download on request, like, a user asks for a movie and it searches, downloads and moves and renames it! If a task is running because of scheduling, it will make that I will have to wait until it ends! My first ideia is to have a symbolic link from the config to a new folder and have it start a new database just for this. And way a link? Because I don't want to duplicate all corrections and improvements in both configs! All works ok, the only thing missing is that in the config I have the schedules, and that will make it run in both instances!

Yeah, that waiting is annoying. I think the best answer would be to start executing that manual task immediately in parallel thread. SQLite is just PITA to deal with that ...maybe we could try it experimentally and see how it goes? Assuming someone is interested adding the support for it :)

I think the best answer would be to start executing that manual task immediately in parallel thread. SQLite is just PITA to deal with that ...maybe we could try it experimentally and see how it goes?

😬


🚨 Potential Security Vulnerability - huntr.dev

⚠ Potential Vulnerability in unset-value
👋 Hello, @jonschlinkert - @arjunshibu has disclosed a potential vulnerability in your repository. We have raised an issue on our fork as patches will first be suggested to our fork before being automatically opened up to your repository. You will receive more information and further instructions about accepting fixes when our community submits their patches. For more information about our process, visit our website.
If you would like to fix the vulnerability yourself, you can do so through our fork and get the bounty rewards! 💸 💵 Make sure to use our PR template when submitting your PR.\r\n\r\nTo view more details about this potential vulnerability, please click here.

☎ Need further support?
Come and join us on our community Discord and a member of our team will be happy to help! 🤗

❓@jonschlinkert - want more notifications like this?
Copy this snippet into your README.md for more vulnerability disclosures and fixes in the future:
[![huntr](https://cdn.huntr.dev/huntr_security_badge_mono.svg)](https://huntr.dev)

cc - @JamieSlome

Before we “fix the vulnerability”, let’s have an expert review it to see if it is one.
Sent from my iPhone

On Apr 9, 2021, at 8:47 AM, huntr.dev | the place to protect open source @.***> wrote:

⚠ Potential Vulnerability in unset-value
👋 Hello, @jonschlinkert - @arjunshibu has disclosed a potential vulnerability in your repository. We have raised an issue on our fork as patches will first be suggested to our fork before being automatically opened up to your repository. You will receive more information and further instructions about accepting fixes when our community submits their patches. For more information about our process, visit our website.
If you would like to fix the vulnerability yourself, you can do so through our fork and get the bounty rewards! 💸 💵 Make sure to use our PR template when submitting your PR.\r\n\r\nTo view more details about this potential vulnerability, please click here.
☎ Need further support?
Come and join us on our community Discord and a member of our team will be happy to help! 🤗
@.*** - want more notifications like this?
Copy this snippet into your README.md for more vulnerability disclosures and fixes in the future:

cc - @JamieSlome
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

I reviewed it, and this is not a vulnerability. Clearly if you do not want someone to delete a protected property you can very easily prevent that in implementation code.
Sent from my iPhone

On Apr 9, 2021, at 8:56 AM, Sellside @.***> wrote:
Before we “fix the vulnerability”, let’s have an expert review it to see if it is one.
Sent from my iPhone

On Apr 9, 2021, at 8:47 AM, huntr.dev | the place to protect open source @.***> wrote:

⚠ Potential Vulnerability in unset-value
👋 Hello, @jonschlinkert - @arjunshibu has disclosed a potential vulnerability in your repository. We have raised an issue on our fork as patches will first be suggested to our fork before being automatically opened up to your repository. You will receive more information and further instructions about accepting fixes when our community submits their patches. For more information about our process, visit our website.
If you would like to fix the vulnerability yourself, you can do so through our fork and get the bounty rewards! 💸 💵 Make sure to use our PR template when submitting your PR.\r\n\r\nTo view more details about this potential vulnerability, please click here.
☎ Need further support?
Come and join us on our community Discord and a member of our team will be happy to help! 🤗
@.*** - want more notifications like this?
Copy this snippet into your README.md for more vulnerability disclosures and fixes in the future:

cc - @JamieSlome
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hi @jonschlinkert , this issue is often an underrated bug. I see that this packge has got   14million+ downloads. So depending on how other projects use this package, there is high chance those projects are vulnerable to permanent DoS attacks. As you said, if someone want to restrict removal of protected methods and properties, they can simply use test cases for it. But the problem is considering this issue is not known for majority of developers, they don't know something like this is possible. So adding a little condition check in this package can fix this issue. You can read this article if interested https://portswigger.net/daily-swig/amp/prototype-pollution-the-dangerous-and-underrated-vulnerability-impacting-javascript-applications
Thanks for your time.

It's underrated? It's a feature of javascript itself:
const obj = {};
console.log('Before:', obj.toString)
delete obj.proto.toString
console.log('After:', obj.toString)
Why would the onus be on this low-level library to prevent implementers
from accepting keys with __proto__? Can you give some examples of how
this would actually be exploited by an end-user in a realistic scenario?
For this be be considered a "vulnerability", I think it's reasonable to
expect an example of at least one implementation where unset-value is used
in a way that would allow this to be exploited.
cc @doowb
On Fri, Apr 9, 2021 at 9:13 AM Arjun Shibu @.***> wrote:

Hi @jonschlinkert https://github.com/jonschlinkert , this issue is
often an underrated bug. I see that this packge has got 14million+
downloads. So depending on how other projects use this package, there is
high chance those projects are vulnerable to permanent DoS attacks. As you
said, if someone want to restrict removal of protected methods and
properties, they can simply use test cases for it. But the problem is
considering this issue is not known for majority of developers, they don't
know something like this is possible. So adding a little condition check in
this package can fix this issue. You can read this article if interested
https://portswigger.net/daily-swig/amp/prototype-pollution-the-dangerous-and-underrated-vulnerability-impacting-javascript-applications
Thanks for your time.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/418sec/unset-value/issues/2#issuecomment-816672456,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAC5X6WZV3MDPFBAZCQXBL3TH34PLANCNFSM42U5DN4Q
.

--
Jon Schlinkert
CEO, Sellside, Inc.
(513) 616-4335
github.com/jonschlinkert
linkedin.com/in/jonschlinkert http://www.linkedin.com/in/jonschlinkert
twitter.com/jonschlinkert http://www.twitter.com/jonschlinkert

Can you give some examples of how this would actually be exploited by an end-user in a realistic scenario?

I would also like to see a realistic scenario where this is exploit exists. The article even states that there aren't many examples of real world cases, so it's hard to see why a low-level library like this should attempt to block a JavaScript feature.


Variable source TS issue with speed/stuttering, can tsp add null packets?

I'm struggling with this and I'm not sure if I can do this at all, so I'd like to ask:

I have live IP source with filtered TS (so it's variable bit rate) or I could even take the full transponder
I want to output this to Dektec DTA-107, for now without further processing, but I want set different modulation parameters than used on the source (for example to convert DVB-S to DVB-S2)
As an parameter for the output, I set symbol rate to 30000000 for example, so it's more than the original stream
Then I tune into this mux on my PC with TBS tuner to analyze the TS - and in the TS, I can see tsp adds null packets, so it looks OK
BUT when I try to play this on my TV, i can see stuttering and errors

Any idea what's missing?

Tsp does not add packet. It can only transform or remove input packets. If you have a variable bitrate input and need to output at a fixed bitrate like DVB-S/S2, tsp is not really suitable for this. If you see null packets on output, this is probably the DTA-107 which is smart enough to add them when its input is starving.
But the output is certainly not smooth. The irregular input probably creates bursts in the output. This is the kind of behaviour a TV or STB does not like. This kind of device expects a smooth real-time stream. PC applications may be more tolerant.

Alright, so are you saying that tsp can be used to remove packets but it won't reduce the TS bitrate since the removed packets must be replaced with null packets in order to keep the constant bit rate of the TS? If that's the case, is another plugin needed for converting vbr mux to cbr? Something like tsvbr2cbr from Opencaster?

No, removed packets are removed, hence reducing the bitrate.
But in your case you want to output a higher bitrate that the input. This would need to create new packets. By the way, I forgot about the option --add-input-stuffing with tsp. Look at the user's manual. This is a way to artificially increase the input bitrate.
However, you still have the issue of the variable input bitrate. The option --add-input-stuffing will increase the bitrate but it will remain variable. But we may use it to increase the bitrate so that the lowest bitrate values are at least as high as the required output bitrate. Then, we would need a plugin that clips at a given bitrate, dropping null packets only. That could work. Just another plugin to develop. Are you volunteer?

Right, I need to output higher bitrate because I don't know the max bitrate of the input but I know it's less than the original TS since it was filtered before reaching tsp. However, TSDuck is already working out for me just fine because I was able to convert source VBR to CBR using tsvbr2cbr tool with just a minor modification in the code of tscbr2vbr (read from stdin instead of looking for a file).
With this command in cygwin, I'm able to watch smooth video on a receiver:
wget -O - http://my_remote_ip:7001 | tsp -I file -O file | ./tsvbr2cbr.exe - 24191000 2> /dev/null | tsp -I file -O dektec --device 0 --frequency<PHONE_NUMBER> -r 7/8 --modulation DVB-S-QPSK --symbol-rate 15000000
Of course it would be nicer to have a plugin that will do this without having to use such workaround but this proves how good and flexible TSDuck already is! I'm sorry but I'm in no way volunteering for any development as I'm not a real developer. I'd love to but I don't have the skills and time.


Add containerName to CallHierarchyItem

Adds a containerName to CallHierarchyItems:

For class methods and accessors (static or instance), uses the class declaration/expression’s name (if it exists)
For object literal methods and accessors, uses the assigned name of the object literal (if it exists)
For top-level function- and class-like declarations in a module declaration, uses the module name (if it is an identifier)

For all other CallHierarchyItems, containerName will be undefined. Note that fileName is already included separately on all items.
Fixes #37061

Do we know what the ux displaying this info in vscode looks like yet?

I personally don’t—@mjbvz or @jrieken might.

Should we have @typescript-bot pack this and let @mjbvz see if this is sufficient for him?

Here's a quick example of how VS Code could render this:

The container name would be shown where src is in this example screenshot


Rename classMap and styleMap to kebab-case

Fixes #641
Feel free to modify the Changelog update or drop it. I wasn't sure if I needed to do that and run gen-docs or not

Thanks @stramel!


SDK-Development doc

Moving the SDK-Development guide to the raiden-ts directory. Almost everything is the same, I started going through the content to phrase it in a more "accessible" way. Note that this was not with the intention to remove anything since the purpose of the document is to provide in-depth technical descriptions of the SDK.
This is an example of the kind of language changes I was striving for:
Before

Raiden client class is the entrypoint for the public API. This class's focus must be Developer Experience (DX), so we should avoid exposing too much of the internals on it, as well as providing anything needed for every reasonable usecase. This balance is important to be kept in mind, and is team's responsibility to decide on whatever non-trivial decision. It's desirable for the interface's changes to be as backwards-compatible as possible, although this isn't guaranteed before MVP release.

After

The Raiden client class is the entrypoint for the public API. The focus of this class is Developer Experience and we should avoid exposing to much of its internals while still providing what is needed for every reasonable use case.

This balance is important to keep in mind. It is desirable that changes in the interface are as backwards compatible as possible, although not necessarily before a MVP release.

As you can see it provides the very same information, but the second version is more digestible in my opinion.
This would be another example:
Before

One of the main development targets is correctness and TypeScript compile-time checks to aid implementation, validation and integration of the diffent parts of the codebase. io-ts is a library which all allows to bridge the gap between TypeScript's compile-time safety and runtime validations. Basically, the problem it solves is: you can tell TypeScript that a function argument, return or a variable is of a given type, and it'll help you not do wrong things with this information (e.g. passing a number to a function which expects a string) while writting your code; but when you're dealing with unknown data (e.g. a handled state file, a incoming message, etc), you can't be sure it matches your expectations, and if not, very bad things can happen.

After

TypeScript helps us check for correctness and aid implementation, validation and integration of various parts of the codebase. We can tell TypeScript what type of argument, return value or variable is expected in a function which helps us avoid passing wrong types when writing our code, like passing a number to a function that expects a string.

However, when we are dealing with unknown data we cannot always be sure it matches our expectations. To bridge this gap we use the io-ts library.

To go through such a dense document as the DSK Development docs is quiet time consuming as I noticed. So for now, this PR moves the document from the Wiki to the repo and through future iterations, we could rework it completely if there is an interest in doing so.

@taleldayekh Simplifying language for the reader always makes sense. Thanks for pointing this out :)
Could you create a separate issue to improve readability of the docs?

Sure thing, I'll do that.


Fix: CI on every branch.

This change is

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Socket.io not serving its .js file with connect

Socket.io doesn't seem to be serving up its socket.io.js file with connect. I swapped connect out for express and everything worked fine.
See https://gist.github.com/1977521

same issue here

@margh with 2.x or 1.x?

2.x for me, it still works in 1.x

probably just needs a few tweaks for 2.x, or browserify

same here, connect 2.x on its own doesn't work but express with connect 1.8.5 works


tests: move coverage code into subsys

move tests/coverage to subsys/testsuite to make cmake runs well when doing coverage enabled sanitycheck
Fixes #13729
Signed-off-by: wentong wu<EMAIL_ADDRESS>

Codecov Report

Merging #13982 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #13982   +/-   ##
=======================================
  Coverage   52.24%   52.24%
=======================================
  Files         307      307
  Lines       45444    45444
  Branches    10511    10511
=======================================
  Hits        23741    23741
  Misses      16911    16911
  Partials     4792     4792

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a44011...3c91fa4. Read the comment docs.

@wentongwu please fix the commit message following the guidelines in https://docs.zephyrproject.org/latest/contribute/#commit-guidelines.

@wentongwu please fix the commit message following the guidelines in https://docs.zephyrproject.org/latest/contribute/#commit-guidelines.

Sorry for the format issue and Thanks for the fix, and I will follow zephyr's submitting proposal.


Model Performance Degradation When Training Llama-2-13b-chat-hf with DPO + deepspeed stage3

Hello, did you meet model Performance Degradation When Training Llama-2-13b-chat-hf with DPO + deepspeed stage3?
I can fintune with Lora.
But using deepspeed stage3 , model performance degrades.

sorry. I find my mistake.


Build failure with Homebrew on Linux

Hello there,
I'm trying to build this project for Homebrew on Linux and I'm getting this output:
https://gist.github.com/dawidd6/9d543b0497edf922bc82d65918d8807a
https://github.com/Homebrew/linuxbrew-core/pull/17644
At this point I'm out of ideas how to solve this.

/usr/bin/ld: /home/linuxbrew/.linuxbrew/lib/libwebsockets.a(unix-plugins.c.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5'

can you verify the installed libwebsockets works?

Yes, it works, just tested a compilation of simple program with brew test libwebsockets.
I think the issue here is that ld wants to link against static build of libwebsockets which is undesirable, cause we provide the shared library too.

I have no idea, it works on homebrew and compile/apt-get installed libwebsockets.
The LIBWEBSOCKETS_LIBRARIES is set to websockets websockets_shared, which will make it always link against static lib first, not sure if it matters.

I think it might be it. Any ideas how to reverse the order without patching sources?

Fixed the issue with this hack:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 4929624..9dd0ea2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -33,13 +33,13 @@ set(LIBWEBSOCKETS_MIN_VERSION 1.7.0)
 set(SOURCE_FILES src/server.c src/http.c src/protocol.c src/utils.c)

 find_package(OpenSSL REQUIRED)
-find_package(Libwebsockets ${LIBWEBSOCKETS_MIN_VERSION} QUIET)
+#find_package(Libwebsockets ${LIBWEBSOCKETS_MIN_VERSION} QUIET)

 find_package(PkgConfig)

-if(Libwebsockets_FOUND)
-    set(LIBWEBSOCKETS_INCLUDE_DIR ${LIBWEBSOCKETS_INCLUDE_DIR} ${LIBWEBSOCKETS_INCLUDE_DIRS})
-else() # try to find libwebsockets with pkg-config
+#if(Libwebsockets_FOUND)
+#    set(LIBWEBSOCKETS_INCLUDE_DIR ${LIBWEBSOCKETS_INCLUDE_DIR} ${LIBWEBSOCKETS_INCLUDE_DIRS})
+#else() # try to find libwebsockets with pkg-config
     pkg_check_modules(Libwebsockets REQUIRED libwebsockets>=${LIBWEBSOCKETS_MIN_VERSION})
     find_path(LIBWEBSOCKETS_INCLUDE_DIR libwebsockets.h
             HINTS ${LIBWEBSOCKETS_INCLUDEDIR} ${LIBWEBSOCKETS_INCLUDE_DIRS})
@@ -48,7 +48,7 @@ else() # try to find libwebsockets with pkg-config
     include(FindPackageHandleStandardArgs)
     find_package_handle_standard_args(LIBWEBSOCKETS DEFAULT_MSG LIBWEBSOCKETS_LIBRARIES LIBWEBSOCKETS_INCLUDE_DIR)
     mark_as_advanced(LIBWEBSOCKETS_INCLUDE_DIR LIBWEBSOCKETS_LIBRARIES)
-endif()
+#endif()

 pkg_check_modules(PC_JSON-C REQUIRED json-c)
 find_path(JSON-C_INCLUDE_DIR json.h


Unexpected marker ID when multilooking

Hello, I have been experimenting with the multilooking feature. For my data set, when I set the multilook parameter to 3 and specify a bounding box, the ariaTSsetup.py routine appears to work through all the given products. However, when I try to apply the next step, MintPy's prep_aria.py, I get an error saying one of the files cannot be read. When I try to look at the extracted unwrapped interferogram that is causing the problem using gdalinfo, I get the following error message:
(ARIA-tools) [rzinke@sar unwrappedPhase]$ gdalinfo 20160903_20160717
ERROR 1: A marker ID was expected (0xff--) instead of 0000cf40
ERROR 1: opj_read_header() failed
gdalinfo failed - unable to open '20160903_20160717'.
This is odd, because all of the other standard products worked in this procedure -- that is, if I toss out that one product, a can run the process from product download to timeseries computation without issue. Furthermore, no errors result when I run the routine with the specified bounding box but without multilooking. Similarly, no errors result when I run the routine without specifying a bounding box but with multilooking. I have deleted and re-downloaded the products for that date, but the error persists.
Has anyone encountered a similar issue in the past? Thanks for any assistance and happy to answer any questions.

@rzinke Based on the discussion you had on GDAL forum it is anticipated behavior for the averaging to have a a pixel shift. The lanczos appeared to remain sharper than the averaging. Based on the two items i understand that lanczos was the recommended re-sampling method. If confirmed and implemented, we can close this ticket.


fix(prow/config): fix tide query config for repo ti-community-infra/configs

Signed-off-by: wuhuizuo<EMAIL_ADDRESS>
What problem does this PR solve?
Issue Number: close #xxx
Problem Summary:
What is changed and how it works?
What's Changed:
How it Works:

/merge


Optimization for pre and post whitespace and selection speed

From<EMAIL_ADDRESS>on December 24, 2012 20:16:32
Great class, can be slow in some cases. However, very helpful. kudos. There are cases where the selection takes longer because of certain node structures. What steps will reproduce the problem? Fiddle shows the problem with an autotest as well http://jsfiddle.net/xFKt8/8/ What is the expected output? What do you see instead? optimized speed for saving ranges on text nodes that are breaks and repeating spaces. What version of the product are you using? On what operating system? CDN dev release for google code. Please provide any additional information below. If you remove the whitespace from the element you will gain a huge performance boost. This does not change the formatting or content style of the element. In my usage it was taking up to 300 ms to get the selection in the dom. If the code handled this case better I would not need to cleanup dom before using your awesome library.
Original issue: http://code.google.com/p/rangy/issues/detail?id=141

Updated jsFiddle: http://jsfiddle.net/xFKt8/9/

Handling the white space just takes time. I don't think there's much I can do about it.


update imported docs

This change is

/assign @steveperry-53 @chenopis

@mengqiy Should we add an entry to _data/reference.yaml, so that this topic shows in the left nav.
Something like this?

title: Cloud Controller Manager
section:

docs/admin/cloud-controller-manager

@mengqiy Do you want someone to do a tech review? Or is a docs review good enough?

Should we add an entry to _data/reference.yaml, so that this topic shows in the left nav.

Good point.
Can you add that in a followup PR, should be trivial. I will be OOO for several days (starting today) with limited availability. So if this PR does have big issue, I suggest to merge it as is.

Do you want someone to do a tech review? Or is a docs review good enough?

I have checked the content should be what I expected.

I have experienced the pain to tweak the update-imported-files.sh to work again. I will definitely rewrite it maybe in go before 1.9 release for sure!


New test cases added

Added several test cases ( opinionatedtest.jl file)

Hi @harinandana02 . Thanks for the PR. Currently the test cases are failing. Could you investigate what's going wrong ? Also, it would be nice if you could provide a single commit (collapsing all your commits into 1) and a small description of what this PR does.

Codecov Report
Patch coverage has no change and project coverage change: +1.56 :tada:

Comparison is base (0c548f1) 59.37% compared to head (6eb9850) 60.93%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main       #5      +/-   ##
==========================================
+ Coverage   59.37%   60.93%   +1.56%
==========================================
  Files           3        3
  Lines         128      128
==========================================
+ Hits           76       78       +2
+ Misses         52       50       -2

Flag
Coverage Δ

unittests
60.93% <ø> (+1.56%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/opinionated.jl
67.05% <ø> (+2.35%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Outdated pull request.


⚠ Bofrost.lu has degraded performance

In fcad4ce, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1780 ms

Resolved: Bofrost.lu performance has improved in 51ed5ed.


Python: Adding DiskANN Vector Index Type to Azure Cosmos DB Mongo vCore

Add a new vector index type diskann to Azure Cosmos DB Mongo vCore vector store. Paper of DiskANN can be found here DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node.

Please see this comment: https://github.com/microsoft/semantic-kernel/pull/9677#issuecomment-2473297168


Add new section

@nish17 this addition doesn't fit well in this file but we can add in another place a list of helpful sites to practice algorithms. Sites like HackerRank, Project Euler, etc.


feat: Clear timer when component is uninstalled

Clear timer when component is uninstalled

Coverage increased (+0.04%) to 82.305% when pulling f33df4481f565851ce93fe79db820465711e12fd on binyellow:master into 15692c337b8236989347d1db3461a67b1271f4d8 on react-component:master.


errors if cargo-dist enabled but haven't yet published with it

"Can't have artifacts header with no artifacts"
You can see this on https://github.com/Gankra/socc-pair
with
{
    "repository":"https://github.com/Gankra/socc-pair",
    "artifacts": { "cargo_dist": true }
}

We should just warn and move on, I think? This isn't the nicest behaviour for someone trying to bring everything up at once who just wants to try things out without running cargo-dist in prod.

This is no longer a crash, but with #382 this situation can be auto-detected and you get completely empty widgets/pages

This would be solved with #397.

This is essentially fixed now. You can get an empty "install" page with this precise config, but that's basically fine if you're going to publish Soon and you can force it off with cargo_dist: false.

Gonna reopen to signify decision on to what extent we should show empty UI to say "yep you enabled this, but we have no content"

We now show this in the empty case

criteria for for-reals closing this: make the widget and install page consistent
By doing exactly one of these two options:

Making the Install Page hide when empty
Making the Widget show when empty

should be finished, test to be sure


Support for Classifier-free ?

Thanks for your impressive work and huge contribution !!!
From your README.md, it can be seen that Classifier Guidance has been supported.
It would be greatful if you could support classifier-free guidance, as it is widely used in many famous work.

Hi @ZGCTroy , thanks for the suggestion. We will add the support and demos in October, including the support for latent diffusion models. (e.g. stable-diffusion).

@ZGCTroy Hi, we've supported classifier-free guidance in the model_wrapper function. Please check our newest version.
If you have any other questions, feel free to ask me :)


NUCLEO-F411RE configuration

NUCLEO STM32F411RET6 BOARD Configuration for TinyCLR

i have some issues with machine and when i saved file it saves wrong files
now i checked all problems and resolves it and saved successfully

downloaded files and tested downloads worked well without problem now so the files with last date of changes are ok

Looks like I didn't notice this PR was targeting master, instead of dev, so I had to revert it. Feel free to open a new PR, targeting dev though, and we can accept it.
We'll also need a signed contributor license agreement from you in order to accept this PR. You can download the CLA form from https://github.com/ghi-electronics/TinyCLR-Ports/blob/dev/README.md#contributing Please sign it, scan and upload it, then return to us at the specified address. You only need to do this once.

sent CLA form to you at specific address


Live Activities Extension 'Undefined symbol'

Hello,
I have already set up the configuration according to your docs, however, when I try to build the iOS app with the extension I get a few errors that suggest there are some issues with Flutter packages.

I use MacBook with M1 and the latest version of XCode (Also I try to run the app on the physical device with the latest iOS version).
What should I do? Have you ever seen these messages when building an app with an extension?
Thank you in advance!

@jakublebiedzinski can you run the example app though?

@mirkancal yup, I could build and run example app

Hum... this is very strange if the example work & not your app !
Can you check if you done all theses steps  ?
Do you tried to clean your Xcode derived data ?
Thanks !

I believe it's connected with the architecture which the app is built with. Has your iOS app been built with x86 architecture? I've noticed that you do not exclude any architectures in the Build Options section in XCode (iOS app).

Thanks for your feedback !
Yes this is possible, you can check the example app, there is an animated countdown timer 👍

I saw that but you use a special function to count down, however, I'd like to count how much time elapsed since a date I provide with shared values and it is not possible with the timerInterval and countDown parameters.

@jakublebiedzinski I think it's also possible to use updateActivity from dart code

Here I have some logs when linking my extension:

Here I have some logs when linking my extension:

Are you sure it's related to live_activities ?

I had to add the Run script in the extension target so it could find the Flutter framework. The script is the the same as in Runner target.


Fix translations

Note that we lost translation for a string due to fa5769e9

Opened by mistake, ignore.


Apple.AirPort version <IP_ADDRESS>

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.4 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

These error in a sandbox, but I believe that might be due to the sandbox being a reduced form of Windows, and not having these bits.


Various improvements to CONTRIBUTING.md

Description

While looking at our CONTRIBUTING.md file, I noticed a number of improvements that could be made:

We should link to the  CONTRIBUTING.md  file in our main repo, as it has a lot of depth to it:
https://github.com/WordPress/openverse/blob/0b5e24bd325da7b75db9484a268d7c8d197ae920/CONTRIBUTING.md
We should remove the reference to CODEBASE.md, which no longer exists: https://github.com/WordPress/openverse-api/blob/main/CONTRIBUTING.md?plain=1#L11
We should add a reference to DOCUMENTATION_GUIDELINES.md: https://github.com/WordPress/openverse-api/blob/429fd45916c9e064ccea772afc184466304bce4e/DOCUMENTATION_GUIDELINES.md
We should remove the line about GitHub discussions (they are currently disabled) in both places its referenced:
https://github.com/WordPress/openverse-api/blob/main/CONTRIBUTING.md?plain=1#L21
https://github.com/WordPress/openverse-api/blob/main/CONTRIBUTING.md?plain=1#L97

Additional context

Inspired by https://github.com/WordPress/openverse-catalog/issues/775
Resolution

[ ] 🙋 I would be interested in resolving this bug.

Hello @AetherUnbound I'm interested in contributing to this issue! Please tell me how to get started.

Hi @Yash-Pratapwar! We'd love to have you help out with this 😄 If you're looking for how to contribute in general, you can take a look at the contributing guide in our main repo! That should have all the instructions for getting started.
For this particular issue, all of the changes we'd like made to the contributing file in this repo are described in the issue description.
I'll assign the issue to you for now 🙂 If you have any more specific questions about contribution, feel free to ask here or in our Make WP Slack!

Thanks @AetherUnbound I have gone through the contributing guide, I had a doubt, whether WSL is required for resolving this issue?

If you're using Windows for your primary development machine, then it's likely that you will need to use WSL in order to run the linting and testing steps. If you're making changes to documentation only, then you may not need to set WSL up to make the changes 🙂 If you're not on Windows (most of the maintainers use OSX or Linux) then you're already set!

Thanks for clearing that up! Will start working on it.
On Tue, Nov 1, 2022, 21:20 Madison Swain-Bowden @.***>
wrote:

If you're using Windows for your primary development machine, then it's
likely that you will need to use WSL in order to run the linting and
testing steps. If you're making changes to documentation only, then you may
not need to set WSL up to make the changes 🙂 If you're not on Windows
(most of the maintainers use OSX or Linux) then you're already set!
—
Reply to this email directly, view it on GitHub
https://github.com/WordPress/openverse-api/issues/973#issuecomment-1298742848,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APWNCAVSE5YKMNRCCGVKZALWGE357ANCNFSM6AAAAAAQ7DDLQA
.
You are receiving this because you were assigned.Message ID:
@.***>

I think we should also replace any commands listed with just recipes where available, e.g. just api-test and just ing-testlocal.

Yes, noted.

@AetherUnbound after removing the line 21 what should I add there instead? I have added a reference to DOCUMENTATION_GUIDELINES.md in that same section, is it ok or should I refrence it in some other section?

I think you could recommend asking a question in the Make WP Slack, similar to what is in the catalog's contributing doc: https://github.com/WordPress/openverse-catalog/commit/3217ed5ecf3f9babeb96929e8883110e5da1c199#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055R23

Description
While looking at our CONTRIBUTING.md file, I noticed a number of improvements that could be made:

We should link to the  CONTRIBUTING.md  file in our main repo, as it has a lot of depth to it:
https://github.com/WordPress/openverse/blob/0b5e24bd325da7b75db9484a268d7c8d197ae920/CONTRIBUTING.md
We should remove the reference to CODEBASE.md, which no longer exists: https://github.com/WordPress/openverse-api/blob/main/CONTRIBUTING.md?plain=1#L11
We should add a reference to DOCUMENTATION_GUIDELINES.md: https://github.com/WordPress/openverse-api/blob/429fd45916c9e064ccea772afc184466304bce4e/DOCUMENTATION_GUIDELINES.md
We should remove the line about GitHub discussions (they are currently disabled) in both places its referenced:
https://github.com/WordPress/openverse-api/blob/main/CONTRIBUTING.md?plain=1#L21
https://github.com/WordPress/openverse-api/blob/main/CONTRIBUTING.md?plain=1#L97

Additional context
Inspired by WordPress/openverse-catalog#775
Resolution

[ ]  🙋 I would be interested in resolving this bug.

@AetherUnbound In the first point you said 'main repo' did you mean this repo (Openverse)?

That's correct!

@AetherUnbound Ok, so for that I would need to fork the Openverse repo then, I did the other 3 tasks for which I will create a PR.


File type migration

After migration of my db to parse server i've found that File type was not migrated at all (_Product table). As the result iOS app returns 403 error when i've tried to call getData method, how to get it?
{
    "_id": "6hCSsc4Nvs",
    "productIdentifier": "com.aksonov.tests.4",
    "icon": "tfss-084db875-a183-441b-9583-5237fdf8d549-0.plist",
    "subtitle": "test",
    "title": "test",
    "order": 4,
    "downloadName": "tfss-481b0948-7e5d-4eb0-aa57-b474a653a219-Husband0.plist",
    "_created_at": {
        "$date": "2014-08-22T05:59:41.458Z"
    },
    "_updated_at": {
        "$date": "2014-08-22T05:59:41.458Z"
    },
    "download": {
        "__type": "File",
        "name": "tfss-481b0948-7e5d-4eb0-aa57-b474a653a219-Husband0.plist",
        "url": "http://files.parsetfss.com/341c3c18-3aea-4b39-8d71-235afe12ecdf/tfss-481b0948-7e5d-4eb0-aa57-b474a653a219-Husband0.plist"
    }
}

@aksonov isn't it the last value of the object you posted, in the download property?

It is value within mlab database. It refers to hosted parse server, that is wrong, right? Also iOS parse client cannot download it - 403 error (maybe because appid is custom, not hosted one)

This is expected, Files are not stored in the database itself in Parse.com hosted apps (see #8).


Update ExtendedBeanSerializer docs

Added note on the columnAnnotation=true configuration parameter requirement. Related to #1868

Yes, I think the note is useful, it should fix future problems.

Removed the trailing space.

Sorry, we're having some issues with Travis.
Could you maybe squash the commits, and add the text [skip ci] to the commit message?

Done.

You can also have the commit message like this:
Added note on required columnAnnotations for ExtendedBeanSerializer

[skip ci]

Sorry for not making that clear

Ok, have changed that.

Can we fetch, rebase and remove the [skip ci]  now, as #1874 has resolved the CI issues we were experiencing.

Rebased against master and removed the [skip ci] - will be closed by https://github.com/querydsl/querydsl/pull/1873

@tomforster Thanks for the input, much appreciated!


Please explain this bit

In Readme,
let recordingsJSON = await response.json();

for (let recording of recordingsJSON) {
... }

Isn't recordingsJSON just a plain object at this point? How is it iterable with for...of ?
Maybe use something like: Object.entries(recordingsJSON)

Yeah, the idea is just that recordingJSON is an iterable array (if it were an object then Object.entries would accomplish the same thing). I've clarified the example a bit to hopefully make that more obvious. Thanks for the question.


Support for default retry

Description
adds 2 default retry on error, first timeout 300ms second after 300 + ~600ms (some randomness)
⚠ Pre-requisite ⚠

[X] TO BE MERGED AFTER https://github.com/apache/openwhisk-client-js/pull/227

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

<EMAIL_ADDRESS>has been released so this PR can be merged. https://www.npmjs.com/package/openwhisk?activeTab=versions
Since it's a patch release, it will be picked up automatically.

The docs on this is:

if retry option is not set on the Openwhisk client,  the default retry is 2 retries, with a first timeout of 200ms (will be ~400ms on the second one).

added docs https://github.com/adobe/aio-lib-runtime/commit/a20a0bc45aa90459f4e529ef1a0af40a89d5d6b2


auth: improve 'removeSubsetRangePerms' to O(n)

Clean up TODOs

lgtm

Thanks @gyuho . BTW, I was planning to rework the range permissions with interval tree (http://www.geeksforgeeks.org/interval-tree/), because it enables O(lg n) lookup for intervals. Do you have any comments or suggestions about the idea (e.g. it's overkill, if you do this, separate repository of the implementation, etc).

@mitake We already have interval tree implementation here https://github.com/coreos/etcd/tree/master/pkg/adt.
/cc @heyitsanthony

@gyuho sorry for my late reply and thanks for your pointer. I didn't notice the package. I'd like to rework the permission checking mechanism after finishing some ongoing auth tasks.


TimeField

In a TimeSpanField, I have a Time field with AM/PM .
In Table, I am having column with Time Datatype.
When I try to save, I get conversion error.   What is the preferred column type in sql server and How to implement in serenity?

You can write your own formatter or in alternative just use a datetime column and format the result you want.

If I have datetime in SQL then, it appends a default date with the time.


Update PDFStreamEngine.java

No need to allocate a new ArrayList here, reduce text extraction time from 16 seconds to 14 seconds on a 4.2M pdf.

This is a read only mirror. Please close and open an issue in JIRA.
https://issues.apache.org/jira/browse/PDFBOX

Of course every speed increase is welcome, but this change is one to be discussed with "the rest of the gang" - what is if one of the processOperator methods keeps the argument list? If not now, maybe at a later time? Your change would pull it under the feet.

@THausherr What do you mean by keep the argument list ? Since you have new ArrayList(), that is exactly the same as clear. I assume you mean someone want to keep the elements in arguments inside processOperator, well, in that case, the clear method only remove elements out of arguments, not destroy them, so if some one keeps reference of the elements, it will still works.

Any progress on this? The users of the passed array must make a copy of the arguments array.

No progress, this is a read only mirror. I told to create an issue in JIRA. I won't create it myself because I'm not persuaded by this. If "The users of the passed array must make a copy of the arguments array." then where would be the speed gain?

I should have written: The users of the passed array, which have to keep a list of the arguments, must make a copy of the arguments array. However I agree, this kind of optimalization must be investigated further, so that there is no unexpected side-effects.
I've created https://github.com/apache/pdfbox/pull/38 which investigates whether the ArrayList is in use after the call to processor. First impression is that this is not the case, and that the optimalization is possible.


/script_1.sh: /bin/bash^M: bad interpreter: No such file or directory

Hi Taner,
Thank for sharing your work. I use all your scripts successfully, all scripts generate great video. I bought VSSG and it generates script_1.sh . When I run script1 in same linux box, it shows issue:
bash: ./script_1.sh: /bin/bash^M: bad interpreter: No such file or directory
Head of script is like following:

START_TIME=$SECONDS
VARIABLE DECLARATIONS
declare ELEMENT_0_FRAME_COUNT=100
declare ELEMENT_0_INCLUDES_AUDIO=
declare ELEMENT_0_DURATION=4
declare ELEMENT_1_FRAME_COUNT=100
declare ELEMENT_1_INCLUDES_AUDIO=
declare ELEMENT_1_DURATION=4
eval "null\ffmpeg -hide_banner -i 'demo/1-test-vid2' -map 0: v: 0 -vf fps=25 -f null - 2>/tmp/.mf.counter"
declare ELEMENT_2_FRAME_COUNT=$(cat -v -e /tmp/.mf.counter | sed 's/.^M//' | grep -Eo "frame=." | grep -Eo ".*fps" | sed 's/fps//g;s/frame//g;s/=//g;s/\ //g')
declare ELEMENT_2_INCLUDES_AUDIO=$(null\ffprobe -i 'demo/1-test-vid2' -show_streams -select_streams a -loglevel error)
declare ELEMENT_2_DURATION=$(null\ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 'demo/1-test-vid2.mp4')
Or the simple script has same issue as well.

Would you please help. Thanks

Hi Taner,
I run file jar in Windows and run sh file in linux docker box. I figure out how your script got issue. The generated script has End of Line characters include CR and LF.
So I use notepad++ to replace CR LF with LF

After that I can run the script.

Would you have a minor update for your program. Thanks.

Hi, thanks for reporting this. I see two issues in your posts.
The first one is about line separator characters. As far as I understand, you have that issue because you are generating the script on Windows but running it on Linux. What happens internally is: application sees that you're running the jar on Windows and generates the script using the line separator characters of it, which are \r\n. And when you run the script on Linux, bash complains because Linux expects to see only \n (\n is the line separator character used by Unix/Linux). I'm not sure how I can prevent this. Application doesn't know where you plan to run the script.
The second issue is about ffmpeg path used in the scripts. Preferences of VSSG allows you to configure where ffmpeg is. By default ffmpeg is used from the system path. VSSG generate the scripts by using ffmpeg only. If you choose to use ffmpeg from another directory you need to select that directory. In that case, scripts will use <directory name>/ffmpeg in the scripts to run ffmpeg. I think, in your case that directory is empty. I assume you wanted to use ffmpeg from another directory to didn't specify it. I can put a warning in the app but it won't solve the problem. It is a configuration error.

Thanks for quick response.


filter on gene in  multiple allele

Add a text  box filter to type in any gene name (or a drop down)  to  filter on any genotypes with said gene.

see https://github.com/pombase/website/issues/250


Feature/issue 73 smpp channel leak

I added some stuff to the base transport to make this work. I also fixed lose_unbound_connection(), which was being annoying in the face of a flapping SMPP connection.

Happy for this to land but discuss your changes with @dmaclay as well.


refactor: Pass object to addParam in iObj functions

Part of #182

Running only the iobj functional tests
$ ./node_modules/mocha/bin/mocha --timeout 3s test/functional/ToolkitFunctional.js

  Toolkit Functional Tests
    retrUsrAuth
      ✓ returns uses's authority for an object using idb tranport (177ms)
      ✓ returns uses's authority for an object using rest tranport (87ms)
      ✓ returns uses's authority for an object using ssh tranport (667ms)
      ✓ returns uses's authority for an object using odbc tranport (133ms)
    rtrCmdInfo
      ✓ returns command info using idb transport (80ms)
      ✓ returns command info using rest transport (80ms)
      ✓ returns command info using ssh transport (688ms)
      ✓ returns command info using odbc transport (83ms)
    retrPgmInfo
      ✓ returns program info using idb transport (101ms)
      ✓ returns program info using rest transport (71ms)
      ✓ returns program info using ssh transport (591ms)
      ✓ returns program info using odbc transport (86ms)
    retrSrvPgmInfo
      ✓ returns service program info using idb transport (139ms)
      ✓ returns service program info using rest transport (73ms)
      ✓ returns service program info using ssh transport (599ms)
      ✓ returns service program info using odbc transport (87ms)
    retrUserInfo
      ✓ returns specified user profile info using idb transport (65ms)
      ✓ returns specified user profile info using rest transport (62ms)
      ✓ returns specified user profile info using ssh transport (639ms)
      ✓ returns specified user profile info using odbc transport (87ms)
    retrUsrAuthToObj
      ✓ retrieves info for users who are authorized to an object using idb transpsort (112ms)
      ✓ retrieves info for users who are authorized to an object using rest transpsort (115ms)
      ✓ retrieves info for users who are authorized to an object using ssh transpsort (692ms)
      ✓ retrieves info for users who are authorized to an object using odbc transpsort (109ms)
    addToLibraryList
      ✓ appends lib to user's lib list using idb transport (131ms)
      ✓ appends lib to user's lib list using rest transport (135ms)
      ✓ appends lib to user's lib list using ssh transport (768ms)
      ✓ appends lib to user's lib list using odbc transport (126ms)

  28 passing (7s)


🐞 Tenant Wide Extensions list not present in App Catalog's Site Contents page

Category

[ ] Question
[ ] Typo
[X] Bug
[ ] Additional article idea

Expected or Desired Behavior
The list Tenant Wide Extensions should appear in the Site Contents page of an app catalog.
Observed Behavior
The list Tenant Wide Extensions is not present in the Site Contents page of an app catalog. This is for a very old tenant (created around the introduction of SPO). Tried both modern & classic experience.

I can access the list directly via the URL...

I can provide the specific tenant privately if it helps troubleshoot.
Steps to Reproduce
Explained above in the Observed Behavior section.

Thanks, @andrewconnell for reporting. List should be indeed visible in the Site Contents as administrators can access that for editing the entries as needed. We'll check the situation on this. This seems to indicate that the list settings are getting for some reason unghosted in your case (old tenant), which explains why it would not be visible, but it's clearly a bug.

This is an unfortunate situation, but most likely what happened is that before the feature was released, the list got unghosted with UI operations. When we then got the final schema.xml files out, they did not have any impact as the list was already unghosted. Technically by design. Inconvenient and should only happen for those scenarios where user knew about this list BEFORE it was released and visible. So - the cost for fixing is too high vs assumed impact to assign resources for this.

:-/
In my case that's not what happened... I never tried going to the list prior to the GA of the feature.

thanks for coming back on this. This would be also an extremely costly thing to fix 100% reliably cross the world and as it's not really blocking issue as such and only in one random tenant, we can't justify the resource ask. Now - if we see this more widely, we'll absolutely need to jump on it.

Np... was just clarifying the situation. Understand risk/reward. Thanks for follow up.


add travis check for trailing whitespace

This adds travis-ci check for trailing whitespace.
Proof: https://travis-ci.org/Marqin/godot/jobs/131918588

Yes, travis check should have failed, because there are broken whitespaces in current code.

I've changed it a little, so test job succeeds it won't build that job further.

I think such checks are a good idea, but it's a bit too early to merge it.
Ideally we should investigate how to setup a client-side git hook that would detect trailing whitespace and notify the committer about it. Those would have to be setup manually by all contributors in their own clones, but if we provide a script to do it and tell them about it when their PRs have whitespace, it should be OK.
So once we have such a script available for all to use, I'd be for merging this PR.

@akien-mga Git's default pre-commit hook checks for trailing whitespace and warns about it. All a user has to do is rename pre-commit.sample to pre-commit in <repo folder>/.git/.

@roastchicken Awesome! We should get this added to some documentation page about coding guidelines (yet non-existing I fear :D).

I will add a complete clang-format test as soon as the Big Reformatting™ is done, so this one won't be needed.


Custom Overrides for PlanExecutions

For some plans, it may make sense to run on an instance with a different value than specified by the Instance or FrameworkVersion, e.g. backup/restore.  An example spec might look like:
apiVersion: maestro.k8s.io/v1alpha1
kind: PlanExecution
metadata:
  name: small-backup
  namespace: default
  ownerReferences:
  - apiVersion: maestro.k8s.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: Instance
    name: small
    uid: a1fc8f64-fa54-11e8-8673-08002795d782
spec:
  instance:
    kind: Instance
    name: small
    namespace: default
  planName: backup
  arguments:
    BACKUP_LOCATION: s3://backup-bucket/data.sql

Some parameters should not be over-rideable and we'd want to include that in the parameter definition in the FrameworkVersion.

I think this is what to some extent KEP-20 will allow