The Higher Education Megathread

Fed up talking videogames? Why?
User avatar
Jenuall
Member
Joined in 2008
AKA: Jenuall
Location: 40 light-years outside of the Exeter nebula
Contact:

PostRe: The Higher Education Megathread
by Jenuall » Tue Dec 28, 2021 1:49 pm

site23 wrote:
Jenuall wrote:I love Pandas, but I'm not sure that will be much help! ;)

Why? It seems like a good fit for the stated problem to me.

It was an attempted joke, don't worry!

User avatar
site23
Member
Joined in 2021

PostRe: The Higher Education Megathread
by site23 » Tue Dec 28, 2021 1:50 pm

Oh OK - my bad, sorry!

Image
Image
User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Tue Dec 28, 2021 2:03 pm

Thanks for the offers of help, much appreciated.

I did do a little research on Python this morning. I have dabbled in it, but I am a novice on it, whilst very confident with Excel. I'm going to try merging data sets in Excel now and see if it can handle the task. My understanding is that pivot tables display all data in a query, even if Excel doesnt display the whole query. I hate pivot tables, but I think there may be a solution there. At least that would take the pressure away from having to cram Python study in to a two week period.

The silly neighbourhood vegetable.
User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Tue Dec 28, 2021 3:24 pm

I am 50/50 on whether this task is achievable within Excel, which is disappointing for something which I picked because it was described as the task which could be completed within Excel. What's really frustrating is that I wouldn't expect to have any issues with the tasks if it was a dataset which actually fits in Excel, but I expect that half of my time will be sat waiting for Excel to calculate / load just due to the volume of data. I've already had it crash on me, just tinkering around and exploring the data.

Pandas is looking more and more likely at this point.

Edit: I have managed to get all of the data in to one combined query. To do this, I had to query all 6 CSV files and append the queries as one. As I normally query databases, this isn't something I've done before, but it was fairly intuitive. The issue was though, that Excel couldn't cope and crashed out, moaning about not having enough memory. Turns out, Office automatically installs the 32 bit version, so I've literally spent the whole day trying to fit this data in to one workbook and have finally achieved it, through reinstalling Office as 64 bit. I'm glad I've looked at this in advance, as this would have been an entire day of my 5 day allocation wasted on pulling the data in to a workbook.

I can still only view the maximum 1,048,541 rows as allowed by Excel, but all 16 million rows can be manipulated within pivot tables, so at very least I know that I can achieve something with the data now...

The silly neighbourhood vegetable.
User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Tue Jan 11, 2022 10:43 am

So, I'm now on day two of my project and I've managed to quite a lot within Python. I've managed to load the CSV files in to dataframes, merge them in to one, create additional fields and manipulate the data by creating pivot tables in Python. I am really pleased that I stuck with Python. Trying to learn it (or at least, learn the bits I needed) in less than two weeks was a big ask, but I've achieved a decent level of data analysis within Python already and hopefully this will be accepted as evidence that I can use a variety of tools. Even if it doesn't, Python is the program I expect to be using in the years to come. I expect to move to Excel for the final output towards the end of the week, giving me 3 or 4 days hands on with Python.

Whilst it has been and will continue to be difficult, I can only imagine how frustrating this task would have been in Excel, where I would likely have spent most of my week faced with system crashes due to the size of the files.

The silly neighbourhood vegetable.
User avatar
Outrunner
Member
Joined in 2008

PostRe: The Higher Education Megathread
by Outrunner » Wed Jan 12, 2022 1:10 pm

Found out today that my application to study abroad for a year has been accepted. I'm going to be in Hong Kong! I'm so excited!

Please do not post this in the "No Context" thread
User avatar
Jenuall
Member
Joined in 2008
AKA: Jenuall
Location: 40 light-years outside of the Exeter nebula
Contact:

PostRe: The Higher Education Megathread
by Jenuall » Wed Jan 12, 2022 1:17 pm

That's exciting Outrunner!

And well done on becoming a Python Pro Sprouty! :toot:

User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Wed Jan 12, 2022 3:15 pm

Jenuall wrote:That's exciting Outrunner!

And well done on becoming a Python Pro Sprouty! :toot:


Thanks! I'm a long way from considering myself a pro at this point, but I am at least evidencing that I have skills beyond my normal toolset, which may well secure me a better grade.

My original intention was to do most of the grunt work within Python, then visualise the data output within Excel. I've come to the conclusion that Excel isn't the right tool for that either, so I am now attempting to visualise within Python. Once again, not my original intention, but it only cements my side goal of proving my ability to use a range of systems. Only downside is that again, I need to learn how to actually do it. :slol:

I have managed to put together a whisker chart within Python, which was one of the two graph types that I did not evidence in my portfolio. I am slightly panicked at this point as I doubt the final output will look as smart as my normal work, but I am hoping the exam board will see my work as a whole and appreciate that whilst it may not look as smart, I have gone to great efforts to do something outside of my comfort zone.

On a personal note, I am pretty pleased that I have actually put together some analysis within Python and look forward to using it further in the future.

The silly neighbourhood vegetable.
User avatar
site23
Member
Joined in 2021

PostRe: The Higher Education Megathread
by site23 » Wed Jan 12, 2022 4:29 pm

Well done! Re: graphics, which plotting library are you using? If you have an hour or so spare you can usually dig into the visual options and make the plots a bit nicer. I use Matplotlib and that can look a bit ragged by default but it has a lot of settings to play with!

Image
Image
User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Wed Jan 12, 2022 7:11 pm

site23 wrote:Well done! Re: graphics, which plotting library are you using? If you have an hour or so spare you can usually dig into the visual options and make the plots a bit nicer. I use Matplotlib and that can look a bit ragged by default but it has a lot of settings to play with!


These are what I have imported, though most of the work is within Pandas. I have used numpy to replace NaN values.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline


Some of the graphs I have used:

Task2b.plot(kind='bar');

Task3.boxplot();

Task5b.plot(kind='bar',xlabel='Actual Cost')


I don't think I have used matplotlib yet, but I obviously had some intention to at some point...

I have no idea how to amend the graphs - colours, formatting, amending labeling etc!

Edit: Chucking a scatter chart into the mix. I know that's matplotlib as I had to import the function.

plt.scatter((Task4b['Actual Cost']),(Task4b['Quantity']),marker='.')
plt.xlabel('Actual Cost (Millions)')
plt.ylabel('Quantity (Millions)');

The silly neighbourhood vegetable.
User avatar
site23
Member
Joined in 2021

PostRe: The Higher Education Megathread
by site23 » Wed Jan 12, 2022 10:19 pm

Sprouty wrote:I have no idea how to amend the graphs - colours, formatting, amending labeling etc!


Here's a Colab notebook with some examples that might be helpful.

https://colab.research.google.com/drive/1EL31gYprKHGf-JIV4bxYrTlEUXqcIeIV?usp=sharing

Image
Image
User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Wed Jan 12, 2022 11:15 pm

site23 wrote:
Sprouty wrote:I have no idea how to amend the graphs - colours, formatting, amending labeling etc!


Here's a Colab notebook with some examples that might be helpful.

https://colab.research.google.com/drive/1EL31gYprKHGf-JIV4bxYrTlEUXqcIeIV?usp=sharing


Appreciated, thank you. I have progressed loads tonight, but the chart type featured in this article is one that needs some tidying, so that looks really helpful.

The silly neighbourhood vegetable.
User avatar
kerr9000
Member
Joined in 2013

PostRe: The Higher Education Megathread
by kerr9000 » Thu Jan 13, 2022 12:57 am

I looked at this for the first proper time just now. I have a Bachelor of Arts in Social Science a 2:2 gotten from Nottingham Trent University, never mentioned it much as I have always been very very sour about getting a 2:2 was 2 marks off a 2:1 and have been bitterly annoyed with myself ever since, but for some reason I think I finally hit a bit of a curve with it and don't feel quiet as cruddy over it

I also have Early Years Professional Status (EYPS) which is a professional status for Early Years Foundation Stage educators (ages 0 – 5), which is intended to be broadly equivalent to Qualified Teacher Status (ages 5 – 18) but targeted to teaching younger kids.

Yet I am a glorified shelf stacker lol

User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Thu Jan 13, 2022 1:58 pm

I have now got to the point that I can produce a presentation from my analysis, all completed within Python. :toot:

There are a couple of things that I would like to add, but if I don't get them in, it wont matter (essentially, adding additional insight beyond the brief). I've put together a merged dataset (15 million records), added fields, created various views using pivot tables and produced nine graphs in the format of pie, scatter, bar, boxplot & line charts.

I would like to add in histograms and a heatmap before putting the final work together in a powerpoint presentation, both are things I think I will be able to achieve.

Thanks for the encouragement & words of advice, much appreciated! :msgreen:

The silly neighbourhood vegetable.
User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Fri Feb 18, 2022 11:31 pm

Quick update and thanks from me.

I received my results this week and passed with a Merit. My coach told me that he felt that doing my final project in Python was probably the thing that secured me the higher grade, so the gamble paid off! I really appreciate the support you guys gave whilst I was trying to crack it.

The silly neighbourhood vegetable.
User avatar
site23
Member
Joined in 2021

PostRe: The Higher Education Megathread
by site23 » Fri Feb 18, 2022 11:36 pm

Many congrats dude, that's brilliant news - sounds like it was very well deserved, well done!

Image
Image
User avatar
aayl1
Sir Aaron of GRcade
Joined in 2008

PostRe: The Higher Education Megathread
by aayl1 » Sat Feb 19, 2022 8:58 am

Nicely done Sprouty!

Was he okay with the project using entirely green text?

Image
User avatar
Sprouty
Member
Joined in 2008
AKA: SillySprout

PostRe: The Higher Education Megathread
by Sprouty » Sat Feb 19, 2022 9:57 am

aayl1 wrote:Nicely done Sprouty!

Was he okay with the project using entirely green text?


:lol: I actually used blue throughout my full portfoilio and the final project and there was a note on my score report criticising my use of an all blue capital font format on my presentation!

Should have used green!

The silly neighbourhood vegetable.
User avatar
Outrunner
Member
Joined in 2008

PostRe: The Higher Education Megathread
by Outrunner » Tue Feb 22, 2022 6:50 pm

I've had my results back for last semesters modules and really happy with what I got. I'm particularly pleased with my Contemporary Japanese Society module which I got 74% on. I loved that module and the tutor was great. I'm now much less worried about keeping my grade average above 60% so I can go to HK.

Please do not post this in the "No Context" thread
User avatar
Rodders9
Member
Joined in 2008

PostRe: The Higher Education Megathread
by Rodders9 » Thu Apr 21, 2022 1:37 pm

Been a while since I posted.

I managed to graduate with a first in Molecular and Cellular Biology MSci (Hons) from The University of Glasgow in 2020.

I'm now 6 months in to a PhD at The University of Strathclyde and all is going well so far.

PSN: Odontochelys

Return to “Stuff”

Who is online

Users browsing this forum: Grumpy David, ITSMILNER, Memento Mori, more heat than light, Neo Cortex, Zilnad and 492 guests