The Very-Many-Questions-Not-Worth-Their-Own-Thread Thread XLII

[warpus]

So there's the answer. It's rich old geezers clutching on to an outdated model.

It's almost comical how they give you access to all this data, but you have to click around to get to it. Obfuscation by design. I thought it smelled fishy. What's surprising is that it took so long for somebody to say: "Oh yeah, it's just old rich geezers trying to squeeze as much money as they can out of this, what did you think"

 

[really]

Bigger exchanges will have better info available to the public generally - it's more likely to be published elsewhere anyways and not as expensive to gather.

 

[Birdjaguar]

Unless you are day trading or the market is rising or falling very quickly, real time data is hardly necessary. Buying Apple at $172.96 versus $173.04 is hardly significant.

 

[MaryKB]

Hey @warpus

I was asking why because understanding what you're really trying to accomplish can help me better visualize what you need

You have to use brokers as facilitators of trade for numerous reasons, many of them dealing with security and integrity. Brokers/traders are licensed and registered, and the purchasing of stocks is regulated. You can go through firms such as with Apps - in those cases what you're doing is not actually buying from the market, but you're buying from what they own. If they need more of a particular stock then they'll buy more. But you definitely couldn't just walk onto the floor of the TSX and start negotiating for purchases. One way you can bypass the exchange of course is through direct purchases - that's where you go through the company issuing the stock (when businesses offer stock purchase plans for employees, this is how they do it)

What you're looking for just doesn't really have much value, which is why you're having such a hard time finding it - this would be something for your own curiosity. Investors look at the market through indexes. The TSX doesn't have a running "master list" because that list can literally change daily. You don't want to look at the companies as just numbers, because you can't really truly compare one stock against another, but rather each stock against itself. You'd want to know what company you're investing in.

I am a data analyst, so I totally understand the desire to want to look at number trends. But I really do feel you're trying to force a square peg into a round hole here.

What is it you want to accomplish with your investments? Do you want to buy cheap (aka penny) stocks and hope that they have a high rate of change quickly, so you can sell and make a quick profit? Are you wanting to try to game the system and jump from stock-to-stock hoping to ride trends? Are you looking for long-term growth?

You can get any company's stock history for free. You can contact a listed company and request a copy of their quarter or annual report. They have to provide this to all existing and potential investors, and they won't charge you for it.

If you're just curious about trending overall stock history, you're really better off using indexes, since the companies within those indexes can come and go. A company could list itself today, sell for a week, then suddenly be out of business and gone (extreme example) You'd have lots of single, detached points. By tracking indexes, you can better observe how different industries are performing over time, and then consider purchasing something from an area you like.

Professional purchasers aren't looking at a list every day of every listing's numbers and making uneducated decisions. They research individual companies, market trends, and other outside influences.

 

[Birdjaguar]

Here is a nice trend chart for the S&P 500 over the past 12 years:

 

[warpus]

Unless you are day trading or the market is rising or falling very quickly, real time data is hardly necessary. Buying Apple at $172.96 versus $173.04 is hardly significant.

This is just how my brain works, so when somebody is hiding data from me I have to ask "Why? How do I get to it?". I agree it's not necessary.

Bigger exchanges will have better info available to the public generally - it's more likely to be published elsewhere anyways and not as expensive to gather.

A slightly different question - do different exchanges have different rules as to who can trade stocks via their exchange? As a Canadian, am I limited to Canadian exchanges only and ETFs are the only way to let's say invest in American or Japanese stocks? Or do I need to hire a broker who specializes in Japanese exchanges and stocks, if that's what I'm after?

I assume that each exchange might have different rules in place, but how does that work exactly? My app only seems to give me access to TSX stocks, so it seems that there might be very well a reason why I am not able to buy let's say New York exchange stocks as well. I know that my pension via work is half invested in American stocks and half in Canadian stocks and bonds, but that all goes through a company that has old geezers in it so they are probably able to do more than a mere clueless investor like me.

Hey @warpus
You have to use brokers as facilitators of trade for numerous reasons, many of them dealing with security and integrity. Brokers/traders are licensed and registered, and the purchasing of stocks is regulated. You can go through firms such as with Apps - in those cases what you're doing is not actually buying from the market, but you're buying from what they own. If they need more of a particular stock then they'll buy more. But you definitely couldn't just walk onto the floor of the TSX and start negotiating for purchases. One way you can bypass the exchange of course is through direct purchases - that's where you go through the company issuing the stock (when businesses offer stock purchase plans for employees, this is how they do it)

Oh, you can buy stocks directly from the company? That's interesting, I didn't know that this was a thing. I'll have to spend some time reading about stocks and exchanges and these decades (centuries?) old systems that are in place, I bet a lot of this will make more sense to me from a historical context. As of now I have like $50 invested in a company I found by sifting through the data and then using that information to research the company to see what it's all about. A trial and a way to get my feet wet a bit, and poke around to see how this all works, but maybe one day I will want to invest some real money, in which case I'd want to have a better understanding of what's going on.

What you're looking for just doesn't really have much value, which is why you're having such a hard time finding it

1. It has value to me and the way I process reality and understand things
2. I have already used my method to identify a stock to buy, even though it was a puny investment
3. This data exists, but is being hidden from me. The TSX does seem to go out of their way to not give me a full list, even though the data is right there - you can see it by searching for individual stocks and accessing their alphabetized list and clicking into each one. It has been explained to me that a full list is generally accessible to brokers and hidden from casual investors like myself. Do you feel that isn't accurate?

You said that this data can change all the time, but the data is there. Right on the TSX website you can get a list of all their stocks that's assembled in a very clunky and inacessible way. Each one has a link. When you click it, you see the live price of the stock, accurate to the last closing date, it seems. The data is there, just never displayed in a full list, or even 90% full list, or anything similar.

It would be simple to assemble this data into a full list. They are already pulling in the price on demand, and are even able to provide "Top 200 stocks this month" lists with all the headings easily.

I understand now that they provide such live and full listings to brokers only, but at first I was like "Huh?". As a programmer who works with data, such a list would be one of the very first things I'd do. Then from there you can narrow it down into more specialized lists. Heck, even monthly full listings would be interesting to have. But nope, they don't give you anything like that.

you can't really truly compare one stock against another, but rather each stock against itself. You'd want to know what company you're investing in.

Yeah, that's why I'd never invest in a company just looking at the numbers and nothing else. I understand the value in researching what you invest in.

I'm not looking to game the system, I just want to see what they have on offer and go from there.

I am a data analyst, so I totally understand the desire to want to look at number trends. But I really do feel you're trying to force a square peg into a round hole here.

"Hey, nice fruit store you have here, do you have a list of all fruit that you have for sale, so I can look through it and see what you have on offer?"
"Nope, stop trying to force a square peg into a round hole"

No offense, but that's what this line of reasoning seems like to me.

What is it you want to accomplish with your investments? Do you want to buy cheap (aka penny) stocks and hope that they have a high rate of change quickly, so you can sell and make a quick profit? Are you wanting to try to game the system and jump from stock-to-stock hoping to ride trends? Are you looking for long-term growth?

I don't have any such goals yet. I don't know how else I can explain that my brain understands numbers better than almost anything.

I had a similar problem when I was looking for a house to buy. Instead of getting the raw data of all the homes for sale and all the details, I had to go through an old geezer who would be able to look this all up for me and pull up what I want. I sucked it up and went along with it, but I didn't like it. Real estate agents seem to have their place, but the way valuable data is hidden from people looking to buy a home just seems like an artificial money grab.

Professional purchasers aren't looking at a list every day of every listing's numbers and making uneducated decisions. They research individual companies, market trends, and other outside influences.

I don't care what professional purchasers do or don't do, and I never said that this was going to be my strategy. I spent time researching the company I ended up investing $50 in, even though it's a meagre amount.

 

[MaryKB]

@warpus I meant it's not considered valuable by professional investors. I totally understand it has a personal worth to you, and I know you're not trying to be like a professional investor. I was trying to illustrate why what you're asking for is very difficult to find or possibly doesn't exist: because you're alone in wanting it.

Data isn't being hidden from you. You can freely get information on any company listed on the exchange. You're demanding that someone provide you a specific type of view that you're looking for. It's possibly something that if you want it you're going to have to build it yourself.

You mistake the exchange for being like a store. This is highly inaccurate. It's more like a bazaar. Any incorporated entity can list itself for sale. The exchange provides a regulated place where this can be done safely and securely. They will show charts and such for major stocks and indexes that are being tracked. But not necessarily for every fly-by-night listing.

You can firm your own corporation and list your shares on the exchange. They're not going to advertise for you. If no one's looking for your company then you're not going to sell much at all .. just a little perhaps to people who come across your listing by accident and don't know what they're doing.

 

[really]

It's not so much that the stock market is holding back. They just don't report on it, or at least timely and easily.

External bodies reporting info like Bloomberg or the original stock ticker were private people providing useful information as a service.

The analogy to the grocer isn't the same as mentioned above, as the stock market isn't offering you an item at a price. Many years ago I worked in a (DIY/Hardware) store - there were price lists available to me in electronic format and if you asked for one I could check it, but if you asked for a complete list of all possible products you would politely be told it wasn't available and be directed to the relevant aisle - there were thousands of items.

https://en.m.wikipedia.org/wiki/Ticker_tape#:~:text=The first stock price ticker,long distance via telegraph wiring.
Edit: on restrictions as a Canadian- if I knew I'd be charging you for that information.

 

[Cutlass]

Numbers is how I make sense of the world. I look at a business magazine that talks about the hottest French cheese company and the top 15 stocks of the week, but it's just words to me. It does not help me understand the big picture. On the other hand, if I look at the whole space represented in numbers - and bam, that's what my brain can understand and make sense of. I can sit down and get to work. I can use all that data to find companies and then research them more.

Does it matter why I want this, anyway? This list must exist, but it is not accessible. I am curious why this is, and how to get my hands on it, or if it's impossible (and why). If only brokers get access to such data, then that would answer my question.

If the vast majority of investors are neurotypicals who like to read words and look at the market that way, then that's fine, but that's not me. I am a numbers guy.



I hope I wasn't too fierce in my questions here, but I often do things my own way, the way it makes sense to my brain. My brain understands numbers, and to properly study the market I need to look at the raw data. If it's incomplete, I start asking "Why? Where can I get the complete data, so I can study this and try to make sense of it?" It seems hidden from view, which I find curious, and was hoping somebody would know why it's done this way. Exchanges charging brokers for the privilege of seeing this data would make sense, if that's indeed what is going on. If it's true, then you'd expect them to go out out of their way to hide this data from everybody else, so that brokers feel like they are paying for something worthwhile, and not just a list they can easily google.

Have you considered going to graduate school for economics? I could maybe introduce you to someone.

 

[Birdjaguar]

@warpus find a way to create what you want and then sell it to investors by subscription. If it is popular, you can quit your job. If it is very popular, you can sell it Bloomberg and retire.

 

[warpus]

You mistake the exchange for being like a store. This is highly inaccurate. It's more like a bazaar. Any incorporated entity can list itself for sale. The exchange provides a regulated place where this can be done safely and securely. They will show charts and such for major stocks and indexes that are being tracked. But not necessarily for every fly-by-night listing.

The store example was definitely not a perfect one, it was mainly meant to highlight some of the head-scratching going on on my end. The head-scratching in question has to do with the fact that this data exists and is easily accessible by the user in various parts of the site, but that any sort of list on display is only a partial view of the information - seemingly hand-crafted from the master list. i.e. Hot cannabis stocks, All stocks that start with A w/ the price suppressed, etc.

I would understand this being a limitation if they were trading millions of stocks; in that case a master list seems like a potential hassle to display for users in a nice convenient format. However:

- There's 2,300 stocks total, a relatively small amount
- They already provide fragments of this data in various places, so it's obvious this data exists and is easily indexable. This can't be denied, you can go and look yourself, every single stock is searchable. A full list of stocks also exists, but is broken up in an artificial way (alphabetically), the price being hidden from view unless you seek it out specifically.
- From what I'm being told brokers have access to this information.

From a data analyst perspective, it's somewhat clear that this organization (i.e. the exchange) is currently going out of their way to not display a full master list to anybody who wants to see it, instead throwing it out in chunks. The reason why this is happening seems to have been answered, so all the pieces seem to connect here for me.

None of this contradicts that professional investors don't need access to such a list. I'm looking at this from the pov of a data analyst. The data is there, displaying it in a list would be a fairly straightforward proposition. Breaking up the data the way they have on various parts of the site and hiding various elements along the way unless you seek them out specifically - this sort of curation takes extra effort, not the other way around.

@warpus find a way to create what you want and then sell it to investors by subscription. If it is popular, you can quit your job. If it is very popular, you can sell it Bloomberg and retire.

To do this I would need access to an API that feeds into this data. I would bet that such a thing does not exist or is tied down in terms of who gets access to it (but I haven't really looked).

i.e. If only brokers have access to such master listings, then it seems the TSX would not just give out API access to their data to me either. And I can't just write a script to scrape their site to pull in all this data on the fly. I mean, I could do that, but they would probably shut me down as soon as they realized what I was doing.

Have you considered going to graduate school for economics? I could maybe introduce you to someone.

I'm a programmer and data analyst, not a doctor.. errr economist.

I don't want to create a new way of investing or whatever. I simply happen to see the world through somewhat different eyes than those who don't see the matrix when they look around them. Nobody else sees the matrix when they look at data, right? I mean, I'm sure there's other weirdos like me out there, but..

My investment methods have by the way so far give me gains of 15% over just 2-3 days. That's a $10 profit! I am now 25 minutes closer to retirement /s

 

[MaryKB]

- From what I'm being told brokers have access to this information.

Brokers create their own lists from the data. If you work for a brokerage, then perhaps their data analysts are constructing something for their brokers to use. The TSX is not producing this report for them that you're looking for.

The reason you can't find it is because no one else wants it, so it hasn't been made. You can easily make this yourself by collecting the data. You want someone else to do the work for you.

The reason they list it in chunks is because that's how people want to look at it. They provide highlights.

While they only have 2,300 listed, those listings change. They're going to have reports for the groupings that most people want to see most of the time. Everything else is available - any company you want to get information on, you can do so.

As a data analyst, you should know that if you want a certain data report in a certain non-standard format, you typically need to build it yourself. I'm not sure why you think this would be different?

There's also no set price for stocks. Let's say Company A stock last sold 10 minutes ago for $52. You buy some now and you pay $55. In half an hour, someone else sells some for $47. Tickers usually show the latest reported sell price (not always current) and compare to the day's open price to look at changes. People follow individual stocks, or indexes.

As someone else suggested, if you think the type of report you're looking for is really valuable, then go build it! You could publish it online, and if there's a huge demand then you could make a lot of money. It'd be a full-time job.

 

[The_J]

Just so that Warpus doesn't think he's crazy: I'm with you man.
If I'd deal in stocks, I'd want to have a downloadable list of all of them, including either last price, average price of last X transactions, or prices of last X transactions. That's a data basis you can work from.

 

[warpus]

As a data analyst, you should know that if you want a certain data report in a certain non-standard format, you typically need to build it yourself. I'm not sure why you think this would be different?

I feel like we are talking past each other in some regards.

My only point here really is that the data is there, living behind the scenes somewhere. It already exists and is being indexed. You can test this yourself.

The full list is also already being assembled (by the TSX), although it is artificially split up into smaller lists, and data points like price are artificially suppressed on these lists as well.

This leads to questions like "Why?".

"Well, nobody ever uses this data except for brokers, and brokers pay to have that access, and that's just how it's always worked and such financial institutions are set in their ways" is an answer to that question, and so far seems like the best answer I've gotten.

 

[Cutlass]

I'm a programmer and data analyst, not a doctor.. errr economist.

I don't want to create a new way of investing or whatever. I simply happen to see the world through somewhat different eyes than those who don't see the matrix when they look around them. Nobody else sees the matrix when they look at data, right? I mean, I'm sure there's other weirdos like me out there, but..

My investment methods have by the way so far give me gains of 15% over just 2-3 days. That's a $10 profit! I am now 25 minutes closer to retirement /s

You'd be surprised how much the economists I talk to do this.

 

[MaryKB]

"Well, nobody ever uses this data except for brokers, and brokers pay to have that access, and that's just how it's always worked and such financial institutions are set in their ways" is an answer to that question, and so far seems like the best answer I've gotten.

That's not the correct answer, which is what I've been trying to say.

 

[Birdjaguar]

My investment methods have by the way so far give me gains of 15% over just 2-3 days. That's a $10 profit! I am now 25 minutes closer to retirement /s

Sell NOW! Put those profits in your pockets before you lose them!

 

[warpus]

That's not the correct answer, which is what I've been trying to say.

I haven't gotten another answer though, that's the thing. "Nobody does it like this so there's no need for such a list to exist" doesn't make much sense, since logistically speaking it would be easier to produce such a list VS the curated lists that the TSX already provides, today, as well as the artificially broken-up-into-sections-that-also-artificially-hides-the-price-and-other-details lists that they also maintain.

They go out of their way to do it this way, it's an extra effort on their part. That's what made me ask questions.

Is it not true that brokers have access to such data? Everything seems to line up that this is indeed a service that only a select few have access to, and that they require certain certifications and that a certain exchange of $$ takes place as well. I thought that maybe there's laws or regulations in place that mandate some of this, but if that were the case surely it'd have come up by now.

In the end it doesn't really matter, as I've pretty much extracted as much information (that will be of use to me personally) out of this as I probably can. It's not like I'm going to study to become a broker, although.. if somebody has information about an open API I can use to extract this data myself, it would be very useful to me. I suspect such APIs do not exist though in a way that's open to the public.

You'd be surprised how much the economists I talk to do this.

They study the data first and foremost and from that master list determine which stocks and companies to research? How do they get access to the master list, is this something they hire somebody to maintain for them, and update manually? Or do they have access to some sort of API or some other way of getting this data? Do they get it via brokers?

 

[Samson]

I am also in the "give me all the data and let me play with it" camp. If someone is hiding something like that I always question their motivations. It cannot be in the interest of either the buyers or sellers of shares.

 

[Cutlass]

They study the data first and foremost and from that master list determine which stocks and companies to research? How do they get access to the master list, is this something they hire somebody to maintain for them, and update manually? Or do they have access to some sort of API or some other way of getting this data? Do they get it via brokers?

I didn't mean about the stock picking. Which economists may do on the side, but that's not their job. What economists do do is look at a bunch of data and see the matrix. And pull an understanding of the world out of the data.